Jul 23 00:02:07.963210 ip-10-0-135-163 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 23 00:02:07.963222 ip-10-0-135-163 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 23 00:02:07.963230 ip-10-0-135-163 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 23 00:02:07.963577 ip-10-0-135-163 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 23 00:02:18.131954 ip-10-0-135-163 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 23 00:02:18.131968 ip-10-0-135-163 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 45f91f1de993412287ad48dfc26989d6 --
Jul 23 00:04:10.172109 ip-10-0-135-163 systemd[1]: Starting Kubernetes Kubelet...
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.511552    2558 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515212    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515222    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515226    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515229    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515232    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515235    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515237    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:04:10.526077 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515240    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515242    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515245    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515247    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515250    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515252    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515255    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515257    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515260    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515263    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515266    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515268    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515271    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515274    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515276    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515279    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515281    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515284    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515286    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515289    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515292    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:04:10.528049 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515296    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515301    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515304    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515308    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515313    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515316    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515319    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515322    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515324    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515327    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515330    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515332    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515335    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515338    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515341    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515344    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515347    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515350    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515352    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515355    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:04:10.530273 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515357    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515360    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515363    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515366    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515369    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515371    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515374    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515377    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515379    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515382    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515384    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515388    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515390    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515393    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515395    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515399    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515403    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515406    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515409    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:04:10.532023 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515412    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515414    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515417    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515419    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515422    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515425    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515427    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515430    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515432    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515435    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515437    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515440    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515443    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515446    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515450    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515454    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515457    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515459    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515462    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:04:10.545006 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515464    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515467    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515469    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515482    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515485    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515489    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515493    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515497    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515501    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515504    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515508    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515512    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.515516    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517817    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517827    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517831    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517852    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517857    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517861    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517865    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:04:10.546795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517869    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517873    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517877    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517880    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517886    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517889    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517893    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517898    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517902    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517906    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517910    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517914    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517918    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517923    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517926    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517930    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517934    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517937    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517941    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517945    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:04:10.549206 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517949    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517953    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517957    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517960    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517964    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517969    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517974    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517978    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517982    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517985    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517989    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517993    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.517996    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518000    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518006    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518010    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518014    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518018    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518025    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518029    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:04:10.560636 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518038    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518042    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518049    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518053    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518057    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518061    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518065    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518068    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518072    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518078    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518083    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518087    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518091    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518095    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518100    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518104    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518109    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518112    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518117    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:04:10.562795 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518123    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518126    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518131    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518134    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518138    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518142    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518146    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518150    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518153    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518158    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518161    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518167    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518171    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518175    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518179    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518183    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518187    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518190    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518195    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518198    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:04:10.564522 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518202    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518207    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518211    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518215    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518218    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518222    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518226    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518230    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518234    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518237    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518241    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518244    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518248    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518756    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518765    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518768    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518773    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518776    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518780    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518784    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:04:10.566781 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518788    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518792    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518796    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518803    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518808    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518812    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518816    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518820    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518824    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518828    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518831    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518852    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518857    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518863    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518867    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518871    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518876    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518880    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518885    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518890    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:04:10.580773 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518894    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518899    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518902    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518904    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518907    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518910    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518912    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518915    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518918    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518921    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518923    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518926    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518928    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518931    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518934    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518938    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518940    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518943    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518945    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518948    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:04:10.583236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518951    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518954    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518957    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518959    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518963    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518967    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518970    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518973    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518975    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518978    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.518982    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519003    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519007    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519010    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519013    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519016    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519019    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519022    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519024    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519027    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:04:10.586291 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519029    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:04:10.584197 ip-10-0-135-163 systemd[1]: Started Kubernetes Kubelet.
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519034    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519037    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519040    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519043    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519045    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519048    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519052    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519055    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519057    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519060    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519062    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519065    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519068    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519070    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519073    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519075    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519078    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519080    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:04:10.602283 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519083    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519085    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519087    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519091    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519093    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519096    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519098    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519101    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519103    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519106    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519108    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519111    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519113    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519182    2558 flags.go:64] FLAG: --address="0.0.0.0"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519200    2558 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519207    2558 flags.go:64] FLAG: --anonymous-auth="true"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519213    2558 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519219    2558 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519222    2558 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519226    2558 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519232    2558 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 23 00:04:10.617827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519236    2558 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519239    2558 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519242    2558 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519245    2558 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519248    2558 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519252    2558 flags.go:64] FLAG: --cgroup-root=""
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519255    2558 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519258    2558 flags.go:64] FLAG: --client-ca-file=""
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519261    2558 flags.go:64] FLAG: --cloud-provider="external"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519263    2558 flags.go:64] FLAG: --cluster-dns="[]"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519268    2558 flags.go:64] FLAG: --cluster-domain=""
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519271    2558 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519274    2558 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519277    2558 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519282    2558 flags.go:64] FLAG: --container-log-max-files="5"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519286    2558 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519289    2558 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519292    2558 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519295    2558 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519298    2558 flags.go:64] FLAG: --contention-profiling="false"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519300    2558 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519303    2558 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519306    2558 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519309    2558 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 23 00:04:10.620644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519313    2558 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519316    2558 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519319    2558 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519321    2558 flags.go:64] FLAG: --enable-load-reader="false"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519324    2558 flags.go:64] FLAG: --enable-server="true"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519327    2558 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519332    2558 flags.go:64] FLAG: --event-burst="100"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519336    2558 flags.go:64] FLAG: --event-qps="50"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519339    2558 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519341    2558 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519345    2558 flags.go:64] FLAG: --eviction-hard=""
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519349    2558 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519351    2558 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519354    2558 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519357    2558 flags.go:64] FLAG: --eviction-soft=""
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519360    2558 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519363    2558 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519366    2558 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519369    2558 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519371    2558 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519374    2558 flags.go:64] FLAG: --fail-swap-on="true"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519377    2558 flags.go:64] FLAG: --feature-gates=""
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519383    2558 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519386    2558 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519389    2558 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 23 00:04:10.622603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519392    2558 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519395    2558 flags.go:64] FLAG: --healthz-port="10248"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519398    2558 flags.go:64] FLAG: --help="false"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519401    2558 flags.go:64] FLAG: --hostname-override="ip-10-0-135-163.ec2.internal"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519404    2558 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519407    2558 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519410    2558 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519414    2558 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519417    2558 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519420    2558 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519423    2558 flags.go:64] FLAG: --image-service-endpoint=""
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519425    2558 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519428    2558 flags.go:64] FLAG: --kube-api-burst="100"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519431    2558 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519436    2558 flags.go:64] FLAG: --kube-api-qps="50"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519439    2558 flags.go:64] FLAG: --kube-reserved=""
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519441    2558 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519444    2558 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519447    2558 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519450    2558 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519453    2558 flags.go:64] FLAG: --lock-file=""
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519455    2558 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519459    2558 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519461    2558 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 23 00:04:10.625192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519467    2558 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519469    2558 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519472    2558 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519475    2558 flags.go:64] FLAG: --logging-format="text"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519478    2558 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519482    2558 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519485    2558 flags.go:64] FLAG: --manifest-url=""
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519488    2558 flags.go:64] FLAG: --manifest-url-header=""
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519492    2558 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519494    2558 flags.go:64] FLAG: --max-open-files="1000000"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519498    2558 flags.go:64] FLAG: --max-pods="110"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519501    2558 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519504    2558 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519507    2558 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519510    2558 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519513    2558 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519516    2558 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519519    2558 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519526    2558 flags.go:64] FLAG: --node-status-max-images="50"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519529    2558 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519532    2558 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519535    2558 flags.go:64] FLAG: --pod-cidr=""
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519539    2558 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0"
Jul 23 00:04:10.828140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519544    2558 flags.go:64] FLAG: --pod-manifest-path=""
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519547    2558 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519550    2558 flags.go:64] FLAG: --pods-per-core="0"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519553    2558 flags.go:64] FLAG: --port="10250"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519564    2558 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519568    2558 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ce76fc65a305bee9"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519571    2558 flags.go:64] FLAG: --qos-reserved=""
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519574    2558 flags.go:64] FLAG: --read-only-port="10255"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519578    2558 flags.go:64] FLAG: --register-node="true"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519580    2558 flags.go:64] FLAG: --register-with-taints=""
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519584    2558 flags.go:64] FLAG: --registry-burst="10"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519587    2558 flags.go:64] FLAG: --registry-qps="5"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519590    2558 flags.go:64] FLAG: --reserved-cpus=""
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519593    2558 flags.go:64] FLAG: --reserved-memory=""
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519597    2558 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519600    2558 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519603    2558 flags.go:64] FLAG: --rotate-certificates="false"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519606    2558 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519609    2558 flags.go:64] FLAG: --runonce="false"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519611    2558 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519614    2558 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519617    2558 flags.go:64] FLAG: --seccomp-default="false"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519620    2558 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519623    2558 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519626    2558 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519629    2558 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 23 00:04:10.850137 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519632    2558 flags.go:64] FLAG: --storage-driver-password="root"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519635    2558 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519638    2558 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519640    2558 flags.go:64] FLAG: --storage-driver-user="root"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519643    2558 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519646    2558 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519650    2558 flags.go:64] FLAG: --system-cgroups=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519653    2558 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519658    2558 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519660    2558 flags.go:64] FLAG: --tls-cert-file=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519663    2558 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519668    2558 flags.go:64] FLAG: --tls-min-version=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519671    2558 flags.go:64] FLAG: --tls-private-key-file=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519673    2558 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519676    2558 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519685    2558 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519688    2558 flags.go:64] FLAG: --v="2"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519697    2558 flags.go:64] FLAG: --version="false"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519701    2558 flags.go:64] FLAG: --vmodule=""
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519704    2558 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.519708    2558 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519818    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519822    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519825    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:04:10.857527 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519828    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519831    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519853    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519856    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519860    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519862    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519865    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519868    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519870    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519873    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519876    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519878    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519881    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519884    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519888    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519890    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519893    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519896    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519898    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519901    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:04:10.858878 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519904    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519906    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519909    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519911    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519915    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519917    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519920    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519922    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519926    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519928    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519931    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519934    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519936    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519939    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519941    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519944    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519946    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519949    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519952    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519954    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:04:10.860158 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519957    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519960    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519962    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519965    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519967    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519970    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519975    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519978    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519982    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519985    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519988    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519990    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519993    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.519996    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520000    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520003    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520006    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520009    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520011    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:04:10.861246 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520014    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520017    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520020    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520023    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520025    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520029    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520032    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520035    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520038    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520040    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520043    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520045    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520048    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520050    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520053    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520055    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520058    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520060    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520063    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:04:10.862121 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520068    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520070    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520073    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520076    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520079    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520081    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520084    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520086    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520089    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520091    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520094    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520096    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520099    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520103    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520106    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520109    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520111    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:04:10.863195 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.520114    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.520730    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.527247    2558 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.527263    2558 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.528074    2558 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.528085    2558 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528144    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528150    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528153    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528156    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528159    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528161    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528164    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528167    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:04:10.864083 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528169    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528172    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528175    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528177    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528180    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528183    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528187    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528191    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528194    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528197    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528200    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528203    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528205    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528208    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528211    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528214    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528216    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528219    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528221    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:04:10.864749 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528226    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528230    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528243    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528246    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528249    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528252    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528255    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528257    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528260    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528262    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528265    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528268    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528271    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528273    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528276    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528278    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528281    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528284    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528287    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:04:10.865645 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528290    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528292    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528295    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528297    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528300    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528302    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528305    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528307    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528310    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528313    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528315    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528318    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528320    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528329    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528331    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528334    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528342    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528345    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528348    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528350    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:04:10.866514 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528353    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528355    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528358    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528361    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528363    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528366    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528368    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528371    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528373    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528377    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528379    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528381    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528384    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528386    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528389    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528391    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528394    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528396    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528399    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528401    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:04:10.867393 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528404    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528406    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528409    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528411    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528414    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528416    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528419    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528423    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528427    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528435    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528438    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528441    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528443    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.528448    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528582    2558 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 23 00:04:10.868271 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528587    2558 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528590    2558 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528593    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528596    2558 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528599    2558 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528601    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528604    2558 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528606    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528609    2558 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528611    2558 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528614    2558 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528616    2558 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528619    2558 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528621    2558 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528624    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528626    2558 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528629    2558 feature_gate.go:328] unrecognized feature gate: Example
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528631    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528634    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528636    2558 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 23 00:04:10.868944 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528639    2558 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528641    2558 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528644    2558 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528646    2558 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528649    2558 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528651    2558 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528654    2558 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528662    2558 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528665    2558 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528668    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528671    2558 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528674    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528676    2558 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528679    2558 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528682    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528685    2558 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528687    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528690    2558 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528692    2558 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 23 00:04:10.869847 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528695    2558 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528697    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528700    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528704    2558 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528707    2558 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528710    2558 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528712    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528715    2558 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528718    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528721    2558 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528723    2558 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528726    2558 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528728    2558 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528731    2558 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528733    2558 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528736    2558 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528738    2558 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528741    2558 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528743    2558 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528746    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 23 00:04:10.870715 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528750    2558 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528758    2558 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528760    2558 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528763    2558 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528766    2558 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528769    2558 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528771    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528774    2558 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528777    2558 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528780    2558 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528782    2558 feature_gate.go:328] unrecognized feature gate: Example2
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528785    2558 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528787    2558 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528790    2558 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528792    2558 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528794    2558 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528797    2558 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528799    2558 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528802    2558 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528804    2558 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 23 00:04:10.871618 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528806    2558 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528809    2558 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528811    2558 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528815    2558 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528818    2558 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528821    2558 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528823    2558 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528826    2558 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528830    2558 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528846    2558 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528850    2558 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528853    2558 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528857    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528860    2558 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528869    2558 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528871    2558 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528874    2558 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528876    2558 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 23 00:04:10.872497 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:10.528879    2558 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.528884    2558 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.528997    2558 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.531006    2558 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.532193    2558 server.go:1019] "Starting client certificate rotation"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.532294    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.532354    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.535318    2558 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.537025    2558 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.546954    2558 log.go:25] "Validated CRI v1 runtime API"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.549195    2558 log.go:25] "Validated CRI v1 image API"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.550056    2558 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 23 00:04:10.873390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.554779    2558 fs.go:135] Filesystem UUIDs: map[0ab08148-6c9c-45a1-a6af-dab60e95323b:/dev/nvme0n1p4 12049383-c102-4ade-9e84-e67f0953f966:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
Jul 23 00:04:10.874046 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.554794    2558 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.560265    2558 manager.go:216] Machine: {Timestamp:2026-07-23 00:04:10.558347729 +0000 UTC m=+0.282662904 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103127 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2a94a219ecfc47fd458657886c6116 SystemUUID:ec2a94a2-19ec-fc47-fd45-8657886c6116 BootID:45f91f1d-e993-4122-87ad-48dfc26989d6 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5a:c5:aa:62:31 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5a:c5:aa:62:31 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:30:49:72:9b:6a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.560357    2558 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.560423    2558 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.561156    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.563120    2558 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.563147    2558 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-163.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 23 00:04:10.874261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.563831    2558 topology_manager.go:138] "Creating topology manager with none policy"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.563852    2558 container_manager_linux.go:309] "Creating device plugin manager"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.563858    2558 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.564515    2558 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.565800    2558 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569404    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569527    2558 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569556    2558 kubelet.go:486] "Attempting to sync node with API server"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569576    2558 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569599    2558 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569632    2558 kubelet.go:398] "Adding apiserver pod source"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.569644    2558 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.570017    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.570040    2558 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.573768    2558 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.576945    2558 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.576973    2558 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578021    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578033    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578038    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578044    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578052    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578057    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578065    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578070    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578077    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578083    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578091    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578097    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578677    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.578683    2558 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.581456    2558 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-163.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.582070    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.582071    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.583404    2558 server.go:1268] "Started kubelet"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.583504    2558 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.583486    2558 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.583676    2558 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.585344    2558 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 23 00:04:10.878294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.585353    2558 server.go:310] "Adding debug handlers to kubelet server"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.588389    2558 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.590848    2558 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.590923    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.590847    2558 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591368    2558 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591372    2558 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591387    2558 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591517    2558 reconstruct.go:97] "Volume reconstruction finished"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591524    2558 reconciler.go:29] "Reconciler: start to sync state"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591831    2558 factory.go:55] Registering systemd factory
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.591861    2558 factory.go:223] Registration of the systemd container factory successfully
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.592039    2558 factory.go:153] Registering CRI-O factory
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.592053    2558 factory.go:223] Registration of the crio container factory successfully
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.592102    2558 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.592125    2558 factory.go:103] Registering Raw factory
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.592141    2558 manager.go:1196] Started watching for new ooms in manager
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.591329    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-163.ec2.internal.18c4c25b7f8de473  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-163.ec2.internal,UID:ip-10-0-135-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-135-163.ec2.internal,},FirstTimestamp:2026-07-23 00:04:10.583377011 +0000 UTC m=+0.307692187,LastTimestamp:2026-07-23 00:04:10.583377011 +0000 UTC m=+0.307692187,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-163.ec2.internal,}"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.592598    2558 manager.go:349] Starting recovery of all containers
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.597908    2558 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.598011    2558 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-135-163.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.598474    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.602780    2558 manager.go:354] Recovery completed
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.607474    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609266    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609289    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609301    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609722    2558 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609731    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609731    2558 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.609762    2558 state_mem.go:36] "Initialized new in-memory state store"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.610378    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.610399    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.051218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.610412    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.611143    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-163.ec2.internal.18c4c25b81191a86  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-163.ec2.internal,UID:ip-10-0-135-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-135-163.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-135-163.ec2.internal,},FirstTimestamp:2026-07-23 00:04:10.609277574 +0000 UTC m=+0.333592748,LastTimestamp:2026-07-23 00:04:10.609277574 +0000 UTC m=+0.333592748,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-163.ec2.internal,}"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.611784    2558 policy_none.go:49] "None policy: Start"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.611796    2558 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.611805    2558 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.613323    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9crnj"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.613735    2558 policy_none.go:47] "Start"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.624947    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9crnj"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.626648    2558 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-135-163.ec2.internal.18c4c25b811959d4  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-135-163.ec2.internal,UID:ip-10-0-135-163.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-135-163.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-135-163.ec2.internal,},FirstTimestamp:2026-07-23 00:04:10.60929378 +0000 UTC m=+0.333608955,LastTimestamp:2026-07-23 00:04:10.60929378 +0000 UTC m=+0.333608955,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-135-163.ec2.internal,}"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.691505    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695428    2558 manager.go:337] "Starting Device Plugin manager"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.695456    2558 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695467    2558 server.go:87] "Starting device plugin registration server"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695692    2558 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695703    2558 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695790    2558 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695920    2558 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.695929    2558 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.696318    2558 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.696362    2558 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.728311    2558 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.728342    2558 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.728358    2558 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.728388    2558 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.730488    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.796757    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.797489    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.059045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.797516    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.797530    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.797555    2558 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.808408    2558 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-163.ec2.internal\" not found" node="ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.810081    2558 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.810096    2558 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-163.ec2.internal\": node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.829228    2558 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"]
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.829292    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.830009    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.830028    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.830039    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833073    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833193    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833229    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833690    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833709    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833718    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833771    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833793    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.833807    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.836788    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.836830    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.837092    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.837479    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.837495    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.837507    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.889494    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-163.ec2.internal\" not found" node="ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.893809    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e08bb259fbd43dc134cebb560b4e3976-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal\" (UID: \"e08bb259fbd43dc134cebb560b4e3976\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.060358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.893830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e08bb259fbd43dc134cebb560b4e3976-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal\" (UID: \"e08bb259fbd43dc134cebb560b4e3976\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.893858    2558 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-163.ec2.internal\" not found" node="ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.893881    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfa0d9d9a7caac4bf13571fee88d8afb-config\") pod \"kube-apiserver-proxy-ip-10-0-135-163.ec2.internal\" (UID: \"bfa0d9d9a7caac4bf13571fee88d8afb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:10.937675    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.994603    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e08bb259fbd43dc134cebb560b4e3976-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal\" (UID: \"e08bb259fbd43dc134cebb560b4e3976\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.994625    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e08bb259fbd43dc134cebb560b4e3976-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal\" (UID: \"e08bb259fbd43dc134cebb560b4e3976\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.994640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfa0d9d9a7caac4bf13571fee88d8afb-config\") pod \"kube-apiserver-proxy-ip-10-0-135-163.ec2.internal\" (UID: \"bfa0d9d9a7caac4bf13571fee88d8afb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.994679    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bfa0d9d9a7caac4bf13571fee88d8afb-config\") pod \"kube-apiserver-proxy-ip-10-0-135-163.ec2.internal\" (UID: \"bfa0d9d9a7caac4bf13571fee88d8afb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.994690    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e08bb259fbd43dc134cebb560b4e3976-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal\" (UID: \"e08bb259fbd43dc134cebb560b4e3976\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:10.994716    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e08bb259fbd43dc134cebb560b4e3976-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal\" (UID: \"e08bb259fbd43dc134cebb560b4e3976\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.061459 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.037865    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.221373 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.140141    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.221373 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.193451    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.221373 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.196571    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:11.404119 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.240875    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.404119 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.341312    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.542251 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.442342    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.542251 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.531670    2558 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 23 00:04:11.542251 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.531769    2558 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 23 00:04:11.543424 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.542894    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.629298    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 23:59:10 +0000 UTC" deadline="2028-03-04 13:33:24.644098817 +0000 UTC"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.629320    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14173h29m13.014782531s"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.643394    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.662516    2558 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.666194    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientMemory"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.666221    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasNoDiskPressure"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.666236    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeHasSufficientPID"
Jul 23 00:04:11.707863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.672163    2558 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:04:11.854635 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.730582    2558 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:04:11.854635 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.744355    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:11.854635 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.844623    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:12.009404 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:11.889962    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfa0d9d9a7caac4bf13571fee88d8afb.slice/crio-53171d4b002b460b029b624c9949fb1b4065ba4bfc92e70d2caa6651999c5f6c WatchSource:0}: Error finding container 53171d4b002b460b029b624c9949fb1b4065ba4bfc92e70d2caa6651999c5f6c: Status 404 returned error can't find the container with id 53171d4b002b460b029b624c9949fb1b4065ba4bfc92e70d2caa6651999c5f6c
Jul 23 00:04:12.009404 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:11.890134    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode08bb259fbd43dc134cebb560b4e3976.slice/crio-7d3be82972dd139f3a6e5eb1736e7e82116bcbe48df01be33162c7e896ad1768 WatchSource:0}: Error finding container 7d3be82972dd139f3a6e5eb1736e7e82116bcbe48df01be33162c7e896ad1768: Status 404 returned error can't find the container with id 7d3be82972dd139f3a6e5eb1736e7e82116bcbe48df01be33162c7e896ad1768
Jul 23 00:04:12.009404 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:11.895123    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:04:12.009404 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:11.945033    2558 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-163.ec2.internal\" not found"
Jul 23 00:04:12.163421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.023505    2558 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:04:12.163421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.091737    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:12.163421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.100475    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 00:04:12.163421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.101814    2558 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal"
Jul 23 00:04:12.163421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.113139    2558 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 23 00:04:12.578031 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.570815    2558 apiserver.go:52] "Watching apiserver"
Jul 23 00:04:12.578031 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.576468    2558 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 23 00:04:12.578031 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.577378    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-fcgjk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal","openshift-multus/multus-additional-cni-plugins-f4llx","openshift-multus/multus-g2p6m","openshift-multus/network-metrics-daemon-csrhk","kube-system/global-pull-secret-syncer-v79jt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5","openshift-image-registry/node-ca-5sfgf","openshift-network-diagnostics/network-check-target-w2rps","openshift-network-operator/iptables-alerter-bx7h7","openshift-ovn-kubernetes/ovnkube-node-72k8d","kube-system/konnectivity-agent-l8l9p","kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal","openshift-cluster-node-tuning-operator/tuned-bmhc8"]
Jul 23 00:04:12.580471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.579656    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.582454    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.585199    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.586711    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.588995    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.589042    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.591182    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.591636    2558 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.593327    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.593406    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.595483    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.598214    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.600065    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602481    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602799    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-env-overrides\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602821    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-cnibin\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602882    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-kubernetes\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602896    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysctl-d\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602916    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-log-socket\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602936    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-hostroot\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.633410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602971    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-socket-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.602991    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-run-netns\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603026    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovn-node-metrics-cert\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603049    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxzvh\" (UniqueName: \"kubernetes.io/projected/976089ec-d588-4575-94ad-b9af93c54f82-kube-api-access-kxzvh\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603071    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-var-lib-kubelet\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-conf-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603099    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-multus-certs\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603117    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt85x\" (UniqueName: \"kubernetes.io/projected/f2d929c9-7b75-4198-9100-ec43735c1037-kube-api-access-tt85x\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603139    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-os-release\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603175    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40fe94b0-8573-4730-a50c-e57954b859d3-multus-daemon-config\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603221    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-systemd-units\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603236    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-systemd\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603252    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-run-ovn-kubernetes\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.638931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603275    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-system-cni-dir\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603289    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-sys\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603314    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-etc-selinux\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603358    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fq5kp\" (UniqueName: \"kubernetes.io/projected/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-kube-api-access-fq5kp\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603379    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysconfig\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603403    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603439    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-system-cni-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603514    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-var-lib-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603550    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovnkube-script-lib\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603574    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7sl\" (UniqueName: \"kubernetes.io/projected/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-kube-api-access-hn7sl\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603597    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysctl-conf\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.639893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603619    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-systemd\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603653    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-cni-multus\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603676    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-registration-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603705    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-kubelet\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603729    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-node-log\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603746    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603770    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-cnibin\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603790    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-etc-kubernetes\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603809    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-ovn\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603850    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603876    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqcxw\" (UniqueName: \"kubernetes.io/projected/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-kube-api-access-tqcxw\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603900    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-etc-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603924    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-cni-binary-copy\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603945    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2d929c9-7b75-4198-9100-ec43735c1037-tmp\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.640486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-cni-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.603986    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-netns\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604008    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604034    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-k8s-cni-cncf-io\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604061    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mdkh\" (UniqueName: \"kubernetes.io/projected/40fe94b0-8573-4730-a50c-e57954b859d3-kube-api-access-5mdkh\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604087    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-device-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604110    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-slash\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604171    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-os-release\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-host\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-cni-bin\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604250    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-sys-fs\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604294    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-cni-bin\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604317    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-lib-modules\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.642525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604383    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-socket-dir-parent\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604414    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40fe94b0-8573-4730-a50c-e57954b859d3-cni-binary-copy\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-cni-netd\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604457    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-run\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604479    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-kubelet\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604502    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovnkube-config\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604534    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604579    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-modprobe-d\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604608    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2d929c9-7b75-4198-9100-ec43735c1037-etc-tuned\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.604769    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.604824    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.616712    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.616747    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.617176    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.617302    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dqndl\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.617347    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.617462    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.618258    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.618282    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.618640    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 23 00:04:12.643325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.618869    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.621370    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624025    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-lljm4\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624035    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624081    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624121    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624025    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-q7vb2\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624225    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 23 00:04:12.645334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.624239    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 23 00:04:12.690081 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690054    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.690895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690607    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.690895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690651    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 23 00:04:12.690895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690650    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-dv6z7\""
Jul 23 00:04:12.690895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690723    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 23 00:04:12.690895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690725    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-csmwd\""
Jul 23 00:04:12.690895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.690820    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.691192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.691099    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-qtzlw\""
Jul 23 00:04:12.691292 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.691276    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.691348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.691311    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zlzs7\""
Jul 23 00:04:12.691521 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.691435    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 23 00:04:12.691593 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.691529    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.691643 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.691599    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.692244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.692227    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-t54gt\""
Jul 23 00:04:12.693120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.692431    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 23 00:04:12.693120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.692469    2558 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 23 00:04:12.693120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.692674    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 23 00:04:12.693120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.692675    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 23 00:04:12.693120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.692817    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-l6w6p\""
Jul 23 00:04:12.703388 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.703369    2558 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705503    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mdkh\" (UniqueName: \"kubernetes.io/projected/40fe94b0-8573-4730-a50c-e57954b859d3-kube-api-access-5mdkh\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705540    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2fc610a8-6e74-4c7f-8822-9db2cf74bda0-agent-certs\") pod \"konnectivity-agent-l8l9p\" (UID: \"2fc610a8-6e74-4c7f-8822-9db2cf74bda0\") " pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705565    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhszj\" (UniqueName: \"kubernetes.io/projected/78c775d2-bca1-4090-837a-078c4a09fb98-kube-api-access-qhszj\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705589    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-device-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705612    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-slash\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705657    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-os-release\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705678    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-host\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705699    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-cni-bin\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705721    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvx7z\" (UniqueName: \"kubernetes.io/projected/7509dbfc-f6e2-46ea-a785-32e381851a89-kube-api-access-gvx7z\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705744    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-sys-fs\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-cni-bin\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-lib-modules\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.707918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-os-release\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705920    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-lib-modules\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705928    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-socket-dir-parent\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705959    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40fe94b0-8573-4730-a50c-e57954b859d3-cni-binary-copy\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.705983    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706005    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-cni-netd\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706033    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-socket-dir-parent\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-run\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706064    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-kubelet\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706075    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-slash\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovnkube-config\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706110    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706126    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-device-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706142    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-modprobe-d\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706143    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.708682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2d929c9-7b75-4198-9100-ec43735c1037-etc-tuned\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706195    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-cni-netd\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706205    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78c775d2-bca1-4090-837a-078c4a09fb98-serviceca\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706229    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/07bb7baf-47aa-4d60-9bf1-881afa525a35-kubelet-config\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706242    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-run\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706255    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-env-overrides\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706279    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-cnibin\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706283    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-kubelet\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706304    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-kubernetes\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706325    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysctl-d\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/07bb7baf-47aa-4d60-9bf1-881afa525a35-dbus\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706382    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-log-socket\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706403    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-hostroot\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78c775d2-bca1-4090-837a-078c4a09fb98-host\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706456    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-socket-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.709348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-run-netns\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706499    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovn-node-metrics-cert\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxzvh\" (UniqueName: \"kubernetes.io/projected/976089ec-d588-4575-94ad-b9af93c54f82-kube-api-access-kxzvh\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706531    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-var-lib-kubelet\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706552    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-conf-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706568    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-multus-certs\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706582    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2fc610a8-6e74-4c7f-8822-9db2cf74bda0-konnectivity-ca\") pod \"konnectivity-agent-l8l9p\" (UID: \"2fc610a8-6e74-4c7f-8822-9db2cf74bda0\") " pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706597    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt85x\" (UniqueName: \"kubernetes.io/projected/f2d929c9-7b75-4198-9100-ec43735c1037-kube-api-access-tt85x\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706617    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-os-release\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706632    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40fe94b0-8573-4730-a50c-e57954b859d3-multus-daemon-config\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7509dbfc-f6e2-46ea-a785-32e381851a89-hosts-file\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706664    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-systemd-units\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-systemd\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706727    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-run-ovn-kubernetes\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.710005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706770    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-system-cni-dir\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706784    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovnkube-config\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706796    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706864    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-sys\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706887    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7509dbfc-f6e2-46ea-a785-32e381851a89-tmp-dir\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706932    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-etc-selinux\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fq5kp\" (UniqueName: \"kubernetes.io/projected/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-kube-api-access-fq5kp\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706987    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysconfig\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707013    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707044    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-modprobe-d\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707048    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-system-cni-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707086    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-host-slash\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707097    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-system-cni-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.710636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-var-lib-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707139    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovnkube-script-lib\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707165    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7sl\" (UniqueName: \"kubernetes.io/projected/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-kube-api-access-hn7sl\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707188    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysctl-conf\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707211    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-systemd\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707236    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-cni-multus\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707259    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-registration-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707283    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-kubelet\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707304    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-var-lib-kubelet\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-node-log\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707344    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-node-log\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707380    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-cnibin\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707408    2558 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.706963    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-tuning-conf-dir\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.711246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-var-lib-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707596    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-sys\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707703    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-kubernetes\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707805    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysctl-d\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707821    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysctl-conf\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707861    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-systemd-units\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.707952    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-os-release\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708028    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-systemd\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708068    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-run-ovn-kubernetes\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708100    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-system-cni-dir\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708164    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovnkube-script-lib\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708168    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-sysconfig\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/976089ec-d588-4575-94ad-b9af93c54f82-cnibin\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708237    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-log-socket\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708282    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-hostroot\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.711904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708369    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-socket-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708411    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-host\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708434    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40fe94b0-8573-4730-a50c-e57954b859d3-multus-daemon-config\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708441    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-etc-kubernetes\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-iptables-alerter-script\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708493    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p76bc\" (UniqueName: \"kubernetes.io/projected/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-kube-api-access-p76bc\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-ovn\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.708520    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708569    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqcxw\" (UniqueName: \"kubernetes.io/projected/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-kube-api-access-tqcxw\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.708604    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:04:13.208568943 +0000 UTC m=+2.932884123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-etc-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708668    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-cni-binary-copy\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708691    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2d929c9-7b75-4198-9100-ec43735c1037-tmp\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.713617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-cni-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708785    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708800    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f2d929c9-7b75-4198-9100-ec43735c1037-etc-systemd\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708802    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-netns\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-etc-selinux\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-cnibin\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708905    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-conf-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708914    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-etc-openvswitch\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708947    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-kubelet\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.708978    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-multus-certs\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709015    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-etc-kubernetes\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-run-ovn\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709111    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709248    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-cni-multus\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.714350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709275    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709303    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-k8s-cni-cncf-io\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709317    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/976089ec-d588-4575-94ad-b9af93c54f82-cni-binary-copy\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709365    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-registration-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709380    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-k8s-cni-cncf-io\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709425    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-kubelet-dir\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709463    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-run-netns\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709474    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-multus-cni-dir\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-host-cni-bin\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709508    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-run-netns\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.709549    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-sys-fs\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.710005    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40fe94b0-8573-4730-a50c-e57954b859d3-cni-binary-copy\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.710070    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-env-overrides\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.710190    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40fe94b0-8573-4730-a50c-e57954b859d3-host-var-lib-cni-bin\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.712537    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.712560    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:12.715005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.712574    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:12.715680 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.712656    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:13.212623846 +0000 UTC m=+2.936939010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:12.715680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.713488    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-ovn-node-metrics-cert\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.715680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.713509    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f2d929c9-7b75-4198-9100-ec43735c1037-etc-tuned\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.715680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.714528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f2d929c9-7b75-4198-9100-ec43735c1037-tmp\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.716548 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.716525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxzvh\" (UniqueName: \"kubernetes.io/projected/976089ec-d588-4575-94ad-b9af93c54f82-kube-api-access-kxzvh\") pod \"multus-additional-cni-plugins-f4llx\" (UID: \"976089ec-d588-4575-94ad-b9af93c54f82\") " pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.716922 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.716904    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mdkh\" (UniqueName: \"kubernetes.io/projected/40fe94b0-8573-4730-a50c-e57954b859d3-kube-api-access-5mdkh\") pod \"multus-g2p6m\" (UID: \"40fe94b0-8573-4730-a50c-e57954b859d3\") " pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.717764 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.717564    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt85x\" (UniqueName: \"kubernetes.io/projected/f2d929c9-7b75-4198-9100-ec43735c1037-kube-api-access-tt85x\") pod \"tuned-bmhc8\" (UID: \"f2d929c9-7b75-4198-9100-ec43735c1037\") " pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.718014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.717975    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7sl\" (UniqueName: \"kubernetes.io/projected/75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a-kube-api-access-hn7sl\") pod \"ovnkube-node-72k8d\" (UID: \"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a\") " pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.718354 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.718274    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fq5kp\" (UniqueName: \"kubernetes.io/projected/1391dc98-f9f9-4c41-bbff-8c7c83ae3e56-kube-api-access-fq5kp\") pod \"aws-ebs-csi-driver-node-59hd5\" (UID: \"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.718560 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.718454    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqcxw\" (UniqueName: \"kubernetes.io/projected/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-kube-api-access-tqcxw\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:12.735105 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.735046    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal" event={"ID":"bfa0d9d9a7caac4bf13571fee88d8afb","Type":"ContainerStarted","Data":"53171d4b002b460b029b624c9949fb1b4065ba4bfc92e70d2caa6651999c5f6c"}
Jul 23 00:04:12.736550 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.736506    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal" event={"ID":"e08bb259fbd43dc134cebb560b4e3976","Type":"ContainerStarted","Data":"7d3be82972dd139f3a6e5eb1736e7e82116bcbe48df01be33162c7e896ad1768"}
Jul 23 00:04:12.737356 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.737320    2558 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 23 00:04:12.738202 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.738185    2558 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qfvzm"
Jul 23 00:04:12.745346 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.745326    2558 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qfvzm"
Jul 23 00:04:12.810076 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2fc610a8-6e74-4c7f-8822-9db2cf74bda0-agent-certs\") pod \"konnectivity-agent-l8l9p\" (UID: \"2fc610a8-6e74-4c7f-8822-9db2cf74bda0\") " pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.810203 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810083    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhszj\" (UniqueName: \"kubernetes.io/projected/78c775d2-bca1-4090-837a-078c4a09fb98-kube-api-access-qhszj\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.810203 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810134    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvx7z\" (UniqueName: \"kubernetes.io/projected/7509dbfc-f6e2-46ea-a785-32e381851a89-kube-api-access-gvx7z\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.810203 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810163    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.810203 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78c775d2-bca1-4090-837a-078c4a09fb98-serviceca\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810214    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/07bb7baf-47aa-4d60-9bf1-881afa525a35-kubelet-config\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810237    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/07bb7baf-47aa-4d60-9bf1-881afa525a35-dbus\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810268    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78c775d2-bca1-4090-837a-078c4a09fb98-host\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810296    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2fc610a8-6e74-4c7f-8822-9db2cf74bda0-konnectivity-ca\") pod \"konnectivity-agent-l8l9p\" (UID: \"2fc610a8-6e74-4c7f-8822-9db2cf74bda0\") " pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810320    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7509dbfc-f6e2-46ea-a785-32e381851a89-hosts-file\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810359    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7509dbfc-f6e2-46ea-a785-32e381851a89-tmp-dir\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-host-slash\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.810431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-iptables-alerter-script\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.810782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810442    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/07bb7baf-47aa-4d60-9bf1-881afa525a35-kubelet-config\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.810782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810448    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p76bc\" (UniqueName: \"kubernetes.io/projected/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-kube-api-access-p76bc\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.810782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810715    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/07bb7baf-47aa-4d60-9bf1-881afa525a35-dbus\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:12.810782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810756    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/78c775d2-bca1-4090-837a-078c4a09fb98-host\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.811002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810795    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7509dbfc-f6e2-46ea-a785-32e381851a89-hosts-file\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.811002 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.810821    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:12.811002 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:12.810894    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:13.310876693 +0000 UTC m=+3.035191877 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:12.811002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.810913    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-host-slash\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.811200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.811078    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7509dbfc-f6e2-46ea-a785-32e381851a89-tmp-dir\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.811496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.811473    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/78c775d2-bca1-4090-837a-078c4a09fb98-serviceca\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.811581 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.811497    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-iptables-alerter-script\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.811581 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.811562    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2fc610a8-6e74-4c7f-8822-9db2cf74bda0-konnectivity-ca\") pod \"konnectivity-agent-l8l9p\" (UID: \"2fc610a8-6e74-4c7f-8822-9db2cf74bda0\") " pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.813958 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.813190    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2fc610a8-6e74-4c7f-8822-9db2cf74bda0-agent-certs\") pod \"konnectivity-agent-l8l9p\" (UID: \"2fc610a8-6e74-4c7f-8822-9db2cf74bda0\") " pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.819804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.819622    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhszj\" (UniqueName: \"kubernetes.io/projected/78c775d2-bca1-4090-837a-078c4a09fb98-kube-api-access-qhszj\") pod \"node-ca-5sfgf\" (UID: \"78c775d2-bca1-4090-837a-078c4a09fb98\") " pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.820305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.820264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p76bc\" (UniqueName: \"kubernetes.io/projected/79fd0eb5-cc78-4dc2-9f71-16888049a7aa-kube-api-access-p76bc\") pod \"iptables-alerter-bx7h7\" (UID: \"79fd0eb5-cc78-4dc2-9f71-16888049a7aa\") " pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:12.820535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.820517    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvx7z\" (UniqueName: \"kubernetes.io/projected/7509dbfc-f6e2-46ea-a785-32e381851a89-kube-api-access-gvx7z\") pod \"node-resolver-fcgjk\" (UID: \"7509dbfc-f6e2-46ea-a785-32e381851a89\") " pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.890969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.890943    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5"
Jul 23 00:04:12.898052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.898031    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:12.905485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.905467    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-f4llx"
Jul 23 00:04:12.912286 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.912263    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-g2p6m"
Jul 23 00:04:12.923133 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.923093    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-bmhc8"
Jul 23 00:04:12.929820 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.929800    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5sfgf"
Jul 23 00:04:12.938446 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.938425    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:12.948278 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.948211    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fcgjk"
Jul 23 00:04:12.987699 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:12.987642    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bx7h7"
Jul 23 00:04:13.106614 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:13.106588    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fd0eb5_cc78_4dc2_9f71_16888049a7aa.slice/crio-e88a27d3dc199df7099fc734b60cad612d7b0a09dab0222b41bd300f33571ff2 WatchSource:0}: Error finding container e88a27d3dc199df7099fc734b60cad612d7b0a09dab0222b41bd300f33571ff2: Status 404 returned error can't find the container with id e88a27d3dc199df7099fc734b60cad612d7b0a09dab0222b41bd300f33571ff2
Jul 23 00:04:13.108306 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:13.108250    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40fe94b0_8573_4730_a50c_e57954b859d3.slice/crio-e159a2df779209b8fa7d0c39b1dbccb977208964a0086298c86ee5f02169c255 WatchSource:0}: Error finding container e159a2df779209b8fa7d0c39b1dbccb977208964a0086298c86ee5f02169c255: Status 404 returned error can't find the container with id e159a2df779209b8fa7d0c39b1dbccb977208964a0086298c86ee5f02169c255
Jul 23 00:04:13.115050 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:13.114966    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c775d2_bca1_4090_837a_078c4a09fb98.slice/crio-9c405c442491d5415eac8a12c0213659f8e105a8cc2935ed004b1b69cc67c1f4 WatchSource:0}: Error finding container 9c405c442491d5415eac8a12c0213659f8e105a8cc2935ed004b1b69cc67c1f4: Status 404 returned error can't find the container with id 9c405c442491d5415eac8a12c0213659f8e105a8cc2935ed004b1b69cc67c1f4
Jul 23 00:04:13.137821 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:13.137792    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75e5c474_9fc2_4a4d_bc80_8f4c5dddc47a.slice/crio-a8c10bbb5847ab6a5639c449afe2b6d545d76af8a40d79e09d19c372bc1d8ec7 WatchSource:0}: Error finding container a8c10bbb5847ab6a5639c449afe2b6d545d76af8a40d79e09d19c372bc1d8ec7: Status 404 returned error can't find the container with id a8c10bbb5847ab6a5639c449afe2b6d545d76af8a40d79e09d19c372bc1d8ec7
Jul 23 00:04:13.140207 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:13.140183    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1391dc98_f9f9_4c41_bbff_8c7c83ae3e56.slice/crio-c4522500f085dfd2ec984211dabacc2b80bb8f7f7ca34306aaa0d3da08db0c27 WatchSource:0}: Error finding container c4522500f085dfd2ec984211dabacc2b80bb8f7f7ca34306aaa0d3da08db0c27: Status 404 returned error can't find the container with id c4522500f085dfd2ec984211dabacc2b80bb8f7f7ca34306aaa0d3da08db0c27
Jul 23 00:04:13.140617 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:13.140597    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7509dbfc_f6e2_46ea_a785_32e381851a89.slice/crio-300a1cd99bcffdd7eb0a4358a3dcec0a0101119bf13c4ceab090d301a99fc6bf WatchSource:0}: Error finding container 300a1cd99bcffdd7eb0a4358a3dcec0a0101119bf13c4ceab090d301a99fc6bf: Status 404 returned error can't find the container with id 300a1cd99bcffdd7eb0a4358a3dcec0a0101119bf13c4ceab090d301a99fc6bf
Jul 23 00:04:13.212817 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.212788    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:13.212964 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.212861    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:13.213027 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.212966    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:13.213027 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.213015    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:13.213119 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.213037    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:04:14.213017232 +0000 UTC m=+3.937332396 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:13.213119 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.213038    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:13.213119 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.213061    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:13.213119 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.213095    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:14.213084579 +0000 UTC m=+3.937399746 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:13.313966 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.313937    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:13.314114 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.314096    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:13.314181 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.314169    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:14.314152336 +0000 UTC m=+4.038467497 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:13.730010 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.729459    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:13.730010 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.729477    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:13.730010 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.729603    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:13.730010 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:13.729724    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:13.744069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.743777    2558 generic.go:356] "Generic (PLEG): container finished" podID="e08bb259fbd43dc134cebb560b4e3976" containerID="8886b9b6e068e07520bf3702b02f7bb65d7538172f259b724c2e8868c71ca2df" exitCode=0
Jul 23 00:04:13.744069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.743878    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal" event={"ID":"e08bb259fbd43dc134cebb560b4e3976","Type":"ContainerDied","Data":"8886b9b6e068e07520bf3702b02f7bb65d7538172f259b724c2e8868c71ca2df"}
Jul 23 00:04:13.746980 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.746217    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"a8c10bbb5847ab6a5639c449afe2b6d545d76af8a40d79e09d19c372bc1d8ec7"}
Jul 23 00:04:13.748669 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.748602    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5sfgf" event={"ID":"78c775d2-bca1-4090-837a-078c4a09fb98","Type":"ContainerStarted","Data":"9c405c442491d5415eac8a12c0213659f8e105a8cc2935ed004b1b69cc67c1f4"}
Jul 23 00:04:13.750152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.750065    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 23:59:12 +0000 UTC" deadline="2028-01-15 23:00:20.648294345 +0000 UTC"
Jul 23 00:04:13.750152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.750108    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13006h56m6.898190713s"
Jul 23 00:04:13.751453 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.751408    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bmhc8" event={"ID":"f2d929c9-7b75-4198-9100-ec43735c1037","Type":"ContainerStarted","Data":"54794d985e3effaacf52675e7dd52b0ca70455d41566999e22f5c5dc7f9b936c"}
Jul 23 00:04:13.758061 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.758035    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bx7h7" event={"ID":"79fd0eb5-cc78-4dc2-9f71-16888049a7aa","Type":"ContainerStarted","Data":"e88a27d3dc199df7099fc734b60cad612d7b0a09dab0222b41bd300f33571ff2"}
Jul 23 00:04:13.761391 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.761329    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fcgjk" event={"ID":"7509dbfc-f6e2-46ea-a785-32e381851a89","Type":"ContainerStarted","Data":"300a1cd99bcffdd7eb0a4358a3dcec0a0101119bf13c4ceab090d301a99fc6bf"}
Jul 23 00:04:13.762703 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.762684    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5" event={"ID":"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56","Type":"ContainerStarted","Data":"c4522500f085dfd2ec984211dabacc2b80bb8f7f7ca34306aaa0d3da08db0c27"}
Jul 23 00:04:13.765641 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.765572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8l9p" event={"ID":"2fc610a8-6e74-4c7f-8822-9db2cf74bda0","Type":"ContainerStarted","Data":"fb958aa502015dff0bd8d239fc9151fe51519277db4df7c6975350359d4bff68"}
Jul 23 00:04:13.768023 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.767948    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerStarted","Data":"d079ae5bfc8c6ef681755f7bf41fdfab59243d2e545546badc7e0df0612fbdeb"}
Jul 23 00:04:13.770039 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:13.769990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g2p6m" event={"ID":"40fe94b0-8573-4730-a50c-e57954b859d3","Type":"ContainerStarted","Data":"e159a2df779209b8fa7d0c39b1dbccb977208964a0086298c86ee5f02169c255"}
Jul 23 00:04:14.222328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.222098    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:14.222442 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.222360    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:14.222799 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.222545    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:14.222799 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.222568    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:14.222799 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.222579    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:14.222799 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.222630    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:16.222614721 +0000 UTC m=+5.946929884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:14.222799 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.222667    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:14.222799 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.222734    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:04:16.222714603 +0000 UTC m=+5.947029766 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:14.323228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.323109    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:14.323380 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.323252    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:14.323380 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.323312    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:16.323294985 +0000 UTC m=+6.047610162 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:14.737928 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.735703    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:14.737928 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:14.735879    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:14.750969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.750907    2558 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 23:59:12 +0000 UTC" deadline="2028-02-04 00:06:11.483374564 +0000 UTC"
Jul 23 00:04:14.750969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.750942    2558 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13464h1m56.732437079s"
Jul 23 00:04:14.779103 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.779068    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal" event={"ID":"bfa0d9d9a7caac4bf13571fee88d8afb","Type":"ContainerStarted","Data":"c098d423f7095015c45f7c915d0895d587b511adda10a3cefd8a5ec992806bcb"}
Jul 23 00:04:14.790280 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.790249    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal" event={"ID":"e08bb259fbd43dc134cebb560b4e3976","Type":"ContainerStarted","Data":"f83d878deb081d54d403c844991dc585b096c045860e667063c3d413cb9d78f9"}
Jul 23 00:04:14.811891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.811327    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-163.ec2.internal" podStartSLOduration=2.811305978 podStartE2EDuration="2.811305978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:14.795977239 +0000 UTC m=+4.520292424" watchObservedRunningTime="2026-07-23 00:04:14.811305978 +0000 UTC m=+4.535621162"
Jul 23 00:04:14.811891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:14.811552    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-163.ec2.internal" podStartSLOduration=2.811539599 podStartE2EDuration="2.811539599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:14.810905141 +0000 UTC m=+4.535220351" watchObservedRunningTime="2026-07-23 00:04:14.811539599 +0000 UTC m=+4.535854788"
Jul 23 00:04:15.729518 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:15.729445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:15.729669 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:15.729593    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:15.729998 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:15.729975    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:15.730104 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:15.730083    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:16.241426 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:16.241392    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:16.241892 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:16.241440    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:16.241892 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.241600    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:16.241892 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.241627    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:16.241892 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.241640    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:16.241892 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.241701    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:20.241682861 +0000 UTC m=+9.965998038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:16.242180 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.242138    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:16.242238 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.242191    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:04:20.242174747 +0000 UTC m=+9.966489915 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:16.342212 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:16.342173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:16.342401 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.342376    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:16.342476 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.342444    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:20.342425292 +0000 UTC m=+10.066740458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:16.728730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:16.728654    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:16.728913 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:16.728794    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:17.729017 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:17.728756    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:17.729017 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:17.728889    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:17.729520 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:17.729277    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:17.729520 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:17.729372    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:18.729545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:18.729511    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:18.729946 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:18.729644    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:19.729360 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:19.729328    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:19.729529 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:19.729466    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:19.729529 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:19.729510    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:19.729961 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:19.729640    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:20.274459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:20.274431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:20.274650 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:20.274476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:20.274650 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.274639    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:20.274764 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.274658    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:20.274764 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.274671    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:20.274764 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.274748    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:28.274708778 +0000 UTC m=+17.999023945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:20.274948 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.274809    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:20.274948 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.274877    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:04:28.274859484 +0000 UTC m=+17.999174660 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:20.375046 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:20.375011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:20.375216 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.375154    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:20.375277 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.375269    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:28.375251586 +0000 UTC m=+18.099566764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:20.730653 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:20.730082    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:20.730653 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:20.730188    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:21.729282 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:21.729241    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:21.729282 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:21.729274    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:21.729512 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:21.729408    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:21.729858 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:21.729809    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:22.729077 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:22.729046    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:22.729461 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:22.729181    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:23.729088 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:23.729052    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:23.729088 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:23.729089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:23.729536 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:23.729173    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:23.729536 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:23.729284    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:24.729639 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:24.729606    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:24.730088 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:24.729721    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:25.729562 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:25.729529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:25.729747 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:25.729661    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:25.729747 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:25.729529    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:25.730180 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:25.730155    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:26.731755 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:26.731721    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:26.732163 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:26.731855    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:27.728657 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:27.728624    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:27.728892 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:27.728744    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:27.728892 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:27.728804    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:27.729014 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:27.728903    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:28.339782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:28.339752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:28.339782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:28.339787    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:28.340255 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.339923    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:28.340255 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.339933    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:28.340255 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.339952    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:28.340255 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.339963    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:28.340255 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.339995    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:04:44.339976032 +0000 UTC m=+34.064291195 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:28.340255 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.340011    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:44.34000465 +0000 UTC m=+34.064319811 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:28.440565 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:28.440539    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:28.440710 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.440654    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:28.440750 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.440711    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:44.440691548 +0000 UTC m=+34.165006720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:28.728664 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:28.728588    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:28.728808 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:28.728715    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:29.728895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:29.728866    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:29.729356 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:29.728903    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:29.729356 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:29.728981    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:29.729356 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:29.729100    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:30.729721 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:30.729689    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:30.730139 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:30.729768    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:31.729241 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:31.729216    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:31.729383 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:31.729316    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:31.729383 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:31.729375    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:31.729468 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:31.729451    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:32.729223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.728930    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:32.729911 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:32.729308    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:32.826530 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.826504    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-g2p6m" event={"ID":"40fe94b0-8573-4730-a50c-e57954b859d3","Type":"ContainerStarted","Data":"fa910a6d509eafbb9800ed5f9f9e53d72581be4acee832df0a40b13910cf8430"}
Jul 23 00:04:32.829082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829063    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:04:32.829367 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829349    2558 generic.go:356] "Generic (PLEG): container finished" podID="75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a" containerID="56666f52ab78637d1e2fbdc17690d5960856ced7dac5c1f340b26f0101c48960" exitCode=1
Jul 23 00:04:32.829432 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829413    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"a7bac35aeae9101271012237c760045f612e44012fa2d47a6069d44997659bbe"}
Jul 23 00:04:32.829488 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829475    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"b2e11c6ef41fc15f9472fa20af09c6c5ad35d69b6f8af94a2a0f2bff438ccae0"}
Jul 23 00:04:32.829542 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829525    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"ee0699631632bb46e2da9fcab0b3341cf6b371ee2ce0a641288e3cdffe84e8db"}
Jul 23 00:04:32.829606 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"22bc79c81ed89627b08fc692d378f42e69be9fd5e58fab3a5ae673ea82958f0e"}
Jul 23 00:04:32.829649 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829628    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerDied","Data":"56666f52ab78637d1e2fbdc17690d5960856ced7dac5c1f340b26f0101c48960"}
Jul 23 00:04:32.829688 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.829664    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"09078762f3f50355a683bfb1486985d627fe38699efbf3ad83b97848f3c712dc"}
Jul 23 00:04:32.830771 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.830747    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5sfgf" event={"ID":"78c775d2-bca1-4090-837a-078c4a09fb98","Type":"ContainerStarted","Data":"c403a356d67b4eebc25265490d8a60bd118e91e72974fef5310e4c2ed20b3aa0"}
Jul 23 00:04:32.833333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.833313    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-bmhc8" event={"ID":"f2d929c9-7b75-4198-9100-ec43735c1037","Type":"ContainerStarted","Data":"6b098e40b0c98724f686ad95e13ea04456956eaac9e342f60896108ba26c63a8"}
Jul 23 00:04:32.834654 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.834629    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fcgjk" event={"ID":"7509dbfc-f6e2-46ea-a785-32e381851a89","Type":"ContainerStarted","Data":"2ae2a8d63541c3d108ad1f46b6c92981df47bc2b946577093f05d61f9075f82e"}
Jul 23 00:04:32.835827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.835808    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5" event={"ID":"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56","Type":"ContainerStarted","Data":"ac0a1a4863962d785ba93f3d4525f5a5362ca7910b7ae86b147bdc908bd3ca0c"}
Jul 23 00:04:32.837141 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.837098    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-l8l9p" event={"ID":"2fc610a8-6e74-4c7f-8822-9db2cf74bda0","Type":"ContainerStarted","Data":"a47d0a9abdd248af549d03eaca66e84f3c90a33fd36c66123c0621dc7b76af00"}
Jul 23 00:04:32.838719 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.838701    2558 generic.go:356] "Generic (PLEG): container finished" podID="976089ec-d588-4575-94ad-b9af93c54f82" containerID="5f7fafdb7579d9d12f9ca12908870a2334ba36e92c4cd66cc9bf381231a24c38" exitCode=0
Jul 23 00:04:32.838831 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.838726    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerDied","Data":"5f7fafdb7579d9d12f9ca12908870a2334ba36e92c4cd66cc9bf381231a24c38"}
Jul 23 00:04:32.845393 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.845354    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-g2p6m" podStartSLOduration=3.7221521810000002 podStartE2EDuration="22.845342922s" totalImagesPullingTime="19.123190741s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:32.843775953 +0000 UTC m=+22.568091149" watchObservedRunningTime="2026-07-23 00:04:32.845342922 +0000 UTC m=+22.569658106"
Jul 23 00:04:32.861366 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.861331    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-bmhc8" podStartSLOduration=4.054311717 podStartE2EDuration="22.861319094s" totalImagesPullingTime="18.807007377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:32.861312335 +0000 UTC m=+22.585627518" watchObservedRunningTime="2026-07-23 00:04:32.861319094 +0000 UTC m=+22.585634277"
Jul 23 00:04:32.944851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.944795    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5sfgf" podStartSLOduration=12.469290538 podStartE2EDuration="22.944781728s" totalImagesPullingTime="10.47549119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:32.924949649 +0000 UTC m=+22.649264859" watchObservedRunningTime="2026-07-23 00:04:32.944781728 +0000 UTC m=+22.669096911"
Jul 23 00:04:32.945051 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.945030    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-l8l9p" podStartSLOduration=4.186439607 podStartE2EDuration="22.945025816s" totalImagesPullingTime="18.758586209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:32.9447516 +0000 UTC m=+22.669066785" watchObservedRunningTime="2026-07-23 00:04:32.945025816 +0000 UTC m=+22.669341000"
Jul 23 00:04:32.963644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:32.963609    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fcgjk" podStartSLOduration=4.18714497 podStartE2EDuration="22.963597629s" totalImagesPullingTime="18.776452659s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:32.963527751 +0000 UTC m=+22.687842947" watchObservedRunningTime="2026-07-23 00:04:32.963597629 +0000 UTC m=+22.687912810"
Jul 23 00:04:33.476590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.476566    2558 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 23 00:04:33.707732 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.707643    2558 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T00:04:33.476585259Z","UUID":"9b554254-b7d1-44f4-8d10-a2efd4dcffdd","Handler":null,"Name":"","Endpoint":""}
Jul 23 00:04:33.709336 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.709318    2558 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 23 00:04:33.709444 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.709368    2558 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 23 00:04:33.728935 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.728910    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:33.729025 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.728910    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:33.729083 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:33.729044    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:33.729083 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:33.729066    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:33.841757 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.841734    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bx7h7" event={"ID":"79fd0eb5-cc78-4dc2-9f71-16888049a7aa","Type":"ContainerStarted","Data":"7d099b43321adc905136a8e2578711186581138a1df9399993e2c553ae652e89"}
Jul 23 00:04:33.843434 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.843409    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5" event={"ID":"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56","Type":"ContainerStarted","Data":"155a458c883936784633cfb79f5abd310cd6a496fd70c6f4b94b9744bf35ae7c"}
Jul 23 00:04:33.857778 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:33.857743    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bx7h7" podStartSLOduration=4.047435392 podStartE2EDuration="22.857732127s" totalImagesPullingTime="18.810296735s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:33.857501523 +0000 UTC m=+23.581816707" watchObservedRunningTime="2026-07-23 00:04:33.857732127 +0000 UTC m=+23.582047351"
Jul 23 00:04:34.729495 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:34.729457    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:34.729646 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:34.729617    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:34.848932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:34.848908    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:04:34.849352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:34.849314    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"1739d662c7d0bbc092f6e1def830fb2ff430d33aa957c06226c752366b86cbe5"}
Jul 23 00:04:34.851598 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:34.851525    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5" event={"ID":"1391dc98-f9f9-4c41-bbff-8c7c83ae3e56","Type":"ContainerStarted","Data":"270279351ba78b65335ed21ac3f7761eeff3bc6d6a6e9e97cb8f5f221ef4a211"}
Jul 23 00:04:34.871755 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:34.871715    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-59hd5" podStartSLOduration=4.035539916 podStartE2EDuration="24.871700953s" totalImagesPullingTime="20.836161037s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:34.871152621 +0000 UTC m=+24.595467805" watchObservedRunningTime="2026-07-23 00:04:34.871700953 +0000 UTC m=+24.596016138"
Jul 23 00:04:35.729035 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:35.728889    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:35.729188 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:35.728888    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:35.729188 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:35.729102    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:35.729188 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:35.729176    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:36.728965 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:36.728935    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:36.729403 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:36.729049    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:37.106535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.106502    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:37.107289 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.107258    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:37.729196 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.729167    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:37.729964 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:37.729275    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:37.729964 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.729347    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:37.729964 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:37.729454    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:37.862123 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.862094    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:04:37.862523 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.862499    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"f9c4c1f6f1c5495229a9993ec5f5cac0e02141bae0f071b02a3d3bf303c25774"}
Jul 23 00:04:37.862684 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.862642    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:37.863180 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.863160    2558 scope.go:117] "RemoveContainer" containerID="56666f52ab78637d1e2fbdc17690d5960856ced7dac5c1f340b26f0101c48960"
Jul 23 00:04:37.863301 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:37.863286    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-l8l9p"
Jul 23 00:04:38.729620 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.729437    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:38.730073 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:38.729788    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:38.739540 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.739515    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v79jt"]
Jul 23 00:04:38.740105 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.740084    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w2rps"]
Jul 23 00:04:38.740184 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.740168    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:38.740248 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:38.740232    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:38.742213 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.742192    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-csrhk"]
Jul 23 00:04:38.742296 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.742271    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:38.742382 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:38.742347    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:38.868273 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.868253    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:04:38.868597 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.868573    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" event={"ID":"75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a","Type":"ContainerStarted","Data":"6e5bc64926dfcd54a0cc5306ecb749c6fe62f2b5004217aa7e4c86390100e8b7"}
Jul 23 00:04:38.868653 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.868602    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:38.868694 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:38.868680    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:38.869122 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.868980    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:38.869122 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.869059    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:38.869122 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.869117    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:38.882810 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.882789    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:38.883035 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.883014    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:04:38.896374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:38.896339    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d" podStartSLOduration=10.078001098 podStartE2EDuration="28.896326545s" totalImagesPullingTime="18.818325447s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:38.895504983 +0000 UTC m=+28.619820170" watchObservedRunningTime="2026-07-23 00:04:38.896326545 +0000 UTC m=+28.620641720"
Jul 23 00:04:39.872766 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:39.872729    2558 generic.go:356] "Generic (PLEG): container finished" podID="976089ec-d588-4575-94ad-b9af93c54f82" containerID="d33cb719e751d02a9b03cd8e4784526680d95c21341a70429ed54d225dac495b" exitCode=0
Jul 23 00:04:39.873182 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:39.872816    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerDied","Data":"d33cb719e751d02a9b03cd8e4784526680d95c21341a70429ed54d225dac495b"}
Jul 23 00:04:40.729325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:40.729297    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:40.729480 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:40.729406    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:40.729543 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:40.729406    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:40.729543 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:40.729517    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:40.729639 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:40.729537    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:40.729673 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:40.729652    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:41.879344 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:41.879306    2558 generic.go:356] "Generic (PLEG): container finished" podID="976089ec-d588-4575-94ad-b9af93c54f82" containerID="fadf39f4faf8995b3c7ce568756c65a59fe2cdc00d1b581f56f1919dba1a7eec" exitCode=0
Jul 23 00:04:41.879800 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:41.879371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerDied","Data":"fadf39f4faf8995b3c7ce568756c65a59fe2cdc00d1b581f56f1919dba1a7eec"}
Jul 23 00:04:42.729222 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:42.729157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:42.729332 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:42.729157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:42.729332 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:42.729294    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:42.729332 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:42.729157    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:42.729460 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:42.729331    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:42.729460 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:42.729399    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:42.883404 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:42.883375    2558 generic.go:356] "Generic (PLEG): container finished" podID="976089ec-d588-4575-94ad-b9af93c54f82" containerID="0a474698ebcf69c7856702bd51c40bb2ba40491a47fe7405acbc057cca3d3e12" exitCode=0
Jul 23 00:04:42.883770 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:42.883430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerDied","Data":"0a474698ebcf69c7856702bd51c40bb2ba40491a47fe7405acbc057cca3d3e12"}
Jul 23 00:04:44.355879 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:44.355828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:44.355886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.355993    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.355995    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.356009    2558 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.356026    2558 projected.go:196] Error preparing data for projected volume kube-api-access-fmj77 for pod openshift-network-diagnostics/network-check-target-w2rps: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.356072    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:05:16.356049865 +0000 UTC m=+66.080365031 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 23 00:04:44.356310 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.356091    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77 podName:08c6dcdf-953e-4610-953b-485350dc7fe9 nodeName:}" failed. No retries permitted until 2026-07-23 00:05:16.356081598 +0000 UTC m=+66.080396767 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-fmj77" (UniqueName: "kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77") pod "network-check-target-w2rps" (UID: "08c6dcdf-953e-4610-953b-485350dc7fe9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 23 00:04:44.456938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:44.456906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:44.457093 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.457058    2558 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:44.457136 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.457104    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret podName:07bb7baf-47aa-4d60-9bf1-881afa525a35 nodeName:}" failed. No retries permitted until 2026-07-23 00:05:16.457091894 +0000 UTC m=+66.181407060 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret") pod "global-pull-secret-syncer-v79jt" (UID: "07bb7baf-47aa-4d60-9bf1-881afa525a35") : object "kube-system"/"original-pull-secret" not registered
Jul 23 00:04:44.732682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:44.732652    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:44.732827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:44.732762    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:44.732827 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.732772    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-v79jt" podUID="07bb7baf-47aa-4d60-9bf1-881afa525a35"
Jul 23 00:04:44.732827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:44.732788    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:44.732983 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.732877    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-w2rps" podUID="08c6dcdf-953e-4610-953b-485350dc7fe9"
Jul 23 00:04:44.732983 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:44.732958    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:04:45.097297 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.097269    2558 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-163.ec2.internal" event="NodeReady"
Jul 23 00:04:45.097440 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.097396    2558 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 23 00:04:45.132522 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.132491    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7cfbc886f-76f7c"]
Jul 23 00:04:45.136042 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.136019    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29746080-c9kg2"]
Jul 23 00:04:45.136185 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.136167    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.138964 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.138944    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 23 00:04:45.139087 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.138977    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 23 00:04:45.139242 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.139227    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 23 00:04:45.139618 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.139599    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8tjsp\""
Jul 23 00:04:45.139943 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.139927    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.142376 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.142348    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-z6zcc\""
Jul 23 00:04:45.142479 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.142457    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\""
Jul 23 00:04:45.151370 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.151343    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 23 00:04:45.152378 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.152265    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29746080-c9kg2"]
Jul 23 00:04:45.154307 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.154278    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cfbc886f-76f7c"]
Jul 23 00:04:45.160649 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.160625    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-l995d"]
Jul 23 00:04:45.164075 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.164059    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ttkgf"]
Jul 23 00:04:45.164201 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.164183    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.166775 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.166754    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 23 00:04:45.166958 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.166939    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 23 00:04:45.167170 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.167155    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-r4q8j\""
Jul 23 00:04:45.167722 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.167705    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.168111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.168093    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 23 00:04:45.170064 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.170045    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6vlrt\""
Jul 23 00:04:45.170164 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.170109    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 23 00:04:45.170164 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.170116    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 23 00:04:45.172741 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.172721    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l995d"]
Jul 23 00:04:45.177463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.176968    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ttkgf"]
Jul 23 00:04:45.263867 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.263825    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dprxm\" (UniqueName: \"kubernetes.io/projected/5a60f4c3-d54a-42ef-93ef-899e4e819daf-kube-api-access-dprxm\") pod \"ingress-canary-l995d\" (UID: \"5a60f4c3-d54a-42ef-93ef-899e4e819daf\") " pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.263999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.263878    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-image-registry-private-configuration\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.263999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.263912    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/55f33850-7ffa-4ed1-83d7-477a646618fa-ca-trust-extracted\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.263999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.263939    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm72c\" (UniqueName: \"kubernetes.io/projected/1b718f23-82f5-4b53-9f6b-e17f42514a82-kube-api-access-dm72c\") pod \"image-pruner-29746080-c9kg2\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") " pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.263999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.263974    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b718f23-82f5-4b53-9f6b-e17f42514a82-serviceca\") pod \"image-pruner-29746080-c9kg2\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") " pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.264200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264018    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.264200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.264200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264094    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqk89\" (UniqueName: \"kubernetes.io/projected/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-kube-api-access-kqk89\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.264200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264133    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-tmp-dir\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.264200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264187    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-certificates\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.264400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-trusted-ca\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.264400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264240    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-bound-sa-token\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.264400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xszkn\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-kube-api-access-xszkn\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.264400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-config-volume\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.264400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a60f4c3-d54a-42ef-93ef-899e4e819daf-cert\") pod \"ingress-canary-l995d\" (UID: \"5a60f4c3-d54a-42ef-93ef-899e4e819daf\") " pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.264400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.264354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-installation-pull-secrets\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365100    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-trusted-ca\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-bound-sa-token\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365338    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xszkn\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-kube-api-access-xszkn\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365388    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-config-volume\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365426    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a60f4c3-d54a-42ef-93ef-899e4e819daf-cert\") pod \"ingress-canary-l995d\" (UID: \"5a60f4c3-d54a-42ef-93ef-899e4e819daf\") " pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365457    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-installation-pull-secrets\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365515    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dprxm\" (UniqueName: \"kubernetes.io/projected/5a60f4c3-d54a-42ef-93ef-899e4e819daf-kube-api-access-dprxm\") pod \"ingress-canary-l995d\" (UID: \"5a60f4c3-d54a-42ef-93ef-899e4e819daf\") " pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365542    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-image-registry-private-configuration\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365577    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/55f33850-7ffa-4ed1-83d7-477a646618fa-ca-trust-extracted\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365601    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm72c\" (UniqueName: \"kubernetes.io/projected/1b718f23-82f5-4b53-9f6b-e17f42514a82-kube-api-access-dm72c\") pod \"image-pruner-29746080-c9kg2\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") " pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.365646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b718f23-82f5-4b53-9f6b-e17f42514a82-serviceca\") pod \"image-pruner-29746080-c9kg2\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") " pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365665    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365706    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365731    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqk89\" (UniqueName: \"kubernetes.io/projected/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-kube-api-access-kqk89\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365768    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-tmp-dir\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.365798    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-certificates\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.366020    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-config-volume\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.366138    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.366154    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.366217    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:04:45.866197818 +0000 UTC m=+35.590512996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.366349    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-trusted-ca\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.366360    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-certificates\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.366392    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b718f23-82f5-4b53-9f6b-e17f42514a82-serviceca\") pod \"image-pruner-29746080-c9kg2\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") " pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.366629    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.366645    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-tmp-dir\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.367052 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.366677    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:45.866661164 +0000 UTC m=+35.590976327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:04:45.367579 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.366701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/55f33850-7ffa-4ed1-83d7-477a646618fa-ca-trust-extracted\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.370157 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.370135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-installation-pull-secrets\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.370271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.370167    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5a60f4c3-d54a-42ef-93ef-899e4e819daf-cert\") pod \"ingress-canary-l995d\" (UID: \"5a60f4c3-d54a-42ef-93ef-899e4e819daf\") " pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.370271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.370181    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-image-registry-private-configuration\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.375591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.375567    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xszkn\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-kube-api-access-xszkn\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.376672 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.376649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm72c\" (UniqueName: \"kubernetes.io/projected/1b718f23-82f5-4b53-9f6b-e17f42514a82-kube-api-access-dm72c\") pod \"image-pruner-29746080-c9kg2\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") " pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.376874 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.376857    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqk89\" (UniqueName: \"kubernetes.io/projected/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-kube-api-access-kqk89\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.376955 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.376892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-bound-sa-token\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.377058 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.377037    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dprxm\" (UniqueName: \"kubernetes.io/projected/5a60f4c3-d54a-42ef-93ef-899e4e819daf-kube-api-access-dprxm\") pod \"ingress-canary-l995d\" (UID: \"5a60f4c3-d54a-42ef-93ef-899e4e819daf\") " pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.456216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.456191    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:04:45.477774 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.477751    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-l995d"
Jul 23 00:04:45.868684 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.868648    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:45.868864 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:45.868702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:45.868864 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.868807    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:04:45.868864 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.868850    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:04:45.868864 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.868865    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:04:45.869050 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.868895    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:46.868877591 +0000 UTC m=+36.593192753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:04:45.869050 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:45.868921    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:04:46.868905936 +0000 UTC m=+36.593221101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:04:46.728959 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.728928    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:04:46.729619 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.728924    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:04:46.729619 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.729111    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:04:46.731461 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.731435    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 00:04:46.732435 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.732369    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 00:04:46.732435 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.732416    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 00:04:46.732640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.732466    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 00:04:46.732640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.732421    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pdllw\""
Jul 23 00:04:46.732640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.732510    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djfbd\""
Jul 23 00:04:46.877282 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.877252    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:46.877407 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:46.877343    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:46.877407 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:46.877394    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:04:46.877491 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:46.877412    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:04:46.877491 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:46.877437    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:04:46.877491 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:46.877460    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:04:48.877446335 +0000 UTC m=+38.601761496 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:04:46.877601 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:46.877496    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:48.877479398 +0000 UTC m=+38.601794565 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:04:47.096545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.096331    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29746080-c9kg2"]
Jul 23 00:04:47.102354 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.102327    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-l995d"]
Jul 23 00:04:47.230869 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:47.230821    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b718f23_82f5_4b53_9f6b_e17f42514a82.slice/crio-5928788d293348118c9c4f67f676df211b6d1113250ca441f994a87efe926ae6 WatchSource:0}: Error finding container 5928788d293348118c9c4f67f676df211b6d1113250ca441f994a87efe926ae6: Status 404 returned error can't find the container with id 5928788d293348118c9c4f67f676df211b6d1113250ca441f994a87efe926ae6
Jul 23 00:04:47.231311 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:04:47.231290    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a60f4c3_d54a_42ef_93ef_899e4e819daf.slice/crio-abb8ef24179a2cb63415963e7689a39b33939a33523061de3c225c5ef2b09bda WatchSource:0}: Error finding container abb8ef24179a2cb63415963e7689a39b33939a33523061de3c225c5ef2b09bda: Status 404 returned error can't find the container with id abb8ef24179a2cb63415963e7689a39b33939a33523061de3c225c5ef2b09bda
Jul 23 00:04:47.897831 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.897785    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29746080-c9kg2" event={"ID":"1b718f23-82f5-4b53-9f6b-e17f42514a82","Type":"ContainerStarted","Data":"f2456160620c9fc1a9c5bd6d67af84891805cd597cc4131d929176af1ac36737"}
Jul 23 00:04:47.898505 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.897862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29746080-c9kg2" event={"ID":"1b718f23-82f5-4b53-9f6b-e17f42514a82","Type":"ContainerStarted","Data":"5928788d293348118c9c4f67f676df211b6d1113250ca441f994a87efe926ae6"}
Jul 23 00:04:47.901358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.901273    2558 generic.go:356] "Generic (PLEG): container finished" podID="976089ec-d588-4575-94ad-b9af93c54f82" containerID="87ad76452bcc036c2990b59a48c4b5f03833065123368915e6899d83c083f16a" exitCode=0
Jul 23 00:04:47.901358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.901309    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerDied","Data":"87ad76452bcc036c2990b59a48c4b5f03833065123368915e6899d83c083f16a"}
Jul 23 00:04:47.902623 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.902602    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l995d" event={"ID":"5a60f4c3-d54a-42ef-93ef-899e4e819daf","Type":"ContainerStarted","Data":"abb8ef24179a2cb63415963e7689a39b33939a33523061de3c225c5ef2b09bda"}
Jul 23 00:04:47.915814 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:47.915778    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-pruner-29746080-c9kg2" podStartSLOduration=287.915761815 podStartE2EDuration="4m47.915761815s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:47.914993974 +0000 UTC m=+37.639309163" watchObservedRunningTime="2026-07-23 00:04:47.915761815 +0000 UTC m=+37.640077000"
Jul 23 00:04:48.895496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:48.895464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:48.895645 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:48.895511    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:48.895645 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:48.895592    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:04:48.895728 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:48.895655    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:04:52.895636551 +0000 UTC m=+42.619951719 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:04:48.895728 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:48.895667    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:04:48.895728 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:48.895685    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:04:48.895867 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:48.895738    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:04:52.895719829 +0000 UTC m=+42.620034990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:04:48.908827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:48.908804    2558 generic.go:356] "Generic (PLEG): container finished" podID="976089ec-d588-4575-94ad-b9af93c54f82" containerID="bf530847114267b4e4eb58daccebe9898168969c78177ab30793406b57b4224f" exitCode=0
Jul 23 00:04:48.909173 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:48.908875    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerDied","Data":"bf530847114267b4e4eb58daccebe9898168969c78177ab30793406b57b4224f"}
Jul 23 00:04:49.915949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:49.915714    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-f4llx" event={"ID":"976089ec-d588-4575-94ad-b9af93c54f82","Type":"ContainerStarted","Data":"cd9fa02166373980cfc53e9a95d9c682137d169abc859366b36a1722d2c32813"}
Jul 23 00:04:49.940463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:49.940416    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-f4llx" podStartSLOduration=9.176940308 podStartE2EDuration="39.94040037s" totalImagesPullingTime="30.763460062s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:49.93870762 +0000 UTC m=+39.663022806" watchObservedRunningTime="2026-07-23 00:04:49.94040037 +0000 UTC m=+39.664715553"
Jul 23 00:04:50.919147 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:50.919110    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-l995d" event={"ID":"5a60f4c3-d54a-42ef-93ef-899e4e819daf","Type":"ContainerStarted","Data":"05649893228cf570f1f9a70e166624b254fb27147c09002fbdc818fb75184fb8"}
Jul 23 00:04:50.936827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:50.936771    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-l995d" podStartSLOduration=2.900948364 podStartE2EDuration="5.936752428s" totalImagesPullingTime="3.035804064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:04:50.935493816 +0000 UTC m=+40.659809024" watchObservedRunningTime="2026-07-23 00:04:50.936752428 +0000 UTC m=+40.661067613"
Jul 23 00:04:52.921648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:52.921612    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:04:52.921648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:04:52.921654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:04:52.922155 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:52.921746    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:04:52.922155 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:52.921791    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:04:52.922155 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:52.921806    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:04:52.922155 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:52.921813    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:05:00.921796799 +0000 UTC m=+50.646111961 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:04:52.922155 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:04:52.921877    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:05:00.921858948 +0000 UTC m=+50.646174131 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:05:00.972542 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:00.972509    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:05:00.972542 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:00.972548    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:05:00.973036 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:00.972649    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:05:00.973036 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:00.972653    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:05:00.973036 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:00.972695    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:05:00.973036 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:00.972699    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:05:16.972685046 +0000 UTC m=+66.697000209 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:05:00.973036 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:00.972736    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:05:16.972723271 +0000 UTC m=+66.697038443 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:05:10.885468 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:10.885435    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-72k8d"
Jul 23 00:05:16.374356 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.374321    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:05:16.374719 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.374364    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:05:16.376709 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.376683    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 23 00:05:16.376810 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.376725    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 23 00:05:16.384664 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.384643    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 23 00:05:16.384761 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.384718    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:06:20.38469648 +0000 UTC m=+130.109011657 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : secret "metrics-daemon-secret" not found
Jul 23 00:05:16.386862 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.386831    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 23 00:05:16.399467 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.399446    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmj77\" (UniqueName: \"kubernetes.io/projected/08c6dcdf-953e-4610-953b-485350dc7fe9-kube-api-access-fmj77\") pod \"network-check-target-w2rps\" (UID: \"08c6dcdf-953e-4610-953b-485350dc7fe9\") " pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:05:16.454290 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.454272    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pdllw\""
Jul 23 00:05:16.462298 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.462281    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:05:16.474779 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.474759    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:05:16.477292 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.477279    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 23 00:05:16.487442 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.487419    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/07bb7baf-47aa-4d60-9bf1-881afa525a35-original-pull-secret\") pod \"global-pull-secret-syncer-v79jt\" (UID: \"07bb7baf-47aa-4d60-9bf1-881afa525a35\") " pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:05:16.614024 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.613993    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-w2rps"]
Jul 23 00:05:16.618765 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:05:16.618738    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08c6dcdf_953e_4610_953b_485350dc7fe9.slice/crio-5be1f35afcac7ecca8e4815b0b843811ef30514cc40e7eae9cae038f7ce1d680 WatchSource:0}: Error finding container 5be1f35afcac7ecca8e4815b0b843811ef30514cc40e7eae9cae038f7ce1d680: Status 404 returned error can't find the container with id 5be1f35afcac7ecca8e4815b0b843811ef30514cc40e7eae9cae038f7ce1d680
Jul 23 00:05:16.746893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.746831    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-v79jt"
Jul 23 00:05:16.889472 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.889448    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-v79jt"]
Jul 23 00:05:16.891723 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:05:16.891697    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bb7baf_47aa_4d60_9bf1_881afa525a35.slice/crio-9a24b39bc90d197289f9d22670127beb4aeb4a3c7858bacd75c70f22946c9353 WatchSource:0}: Error finding container 9a24b39bc90d197289f9d22670127beb4aeb4a3c7858bacd75c70f22946c9353: Status 404 returned error can't find the container with id 9a24b39bc90d197289f9d22670127beb4aeb4a3c7858bacd75c70f22946c9353
Jul 23 00:05:16.978782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.978761    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:05:16.978900 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.978794    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:05:16.978944 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.978916    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:05:16.978944 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.978922    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:05:16.979004 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.978988    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:05:48.978965487 +0000 UTC m=+98.703280652 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:05:16.979047 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.978927    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:05:16.979077 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:16.979047    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:05:48.979035862 +0000 UTC m=+98.703351029 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:05:16.980190 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.980167    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v79jt" event={"ID":"07bb7baf-47aa-4d60-9bf1-881afa525a35","Type":"ContainerStarted","Data":"9a24b39bc90d197289f9d22670127beb4aeb4a3c7858bacd75c70f22946c9353"}
Jul 23 00:05:16.981249 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:16.981229    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w2rps" event={"ID":"08c6dcdf-953e-4610-953b-485350dc7fe9","Type":"ContainerStarted","Data":"5be1f35afcac7ecca8e4815b0b843811ef30514cc40e7eae9cae038f7ce1d680"}
Jul 23 00:05:19.990648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:19.990603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-w2rps" event={"ID":"08c6dcdf-953e-4610-953b-485350dc7fe9","Type":"ContainerStarted","Data":"3f782572f35cb8253951b067751688e96158cc01529033967a6804b44439c18c"}
Jul 23 00:05:19.991072 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:19.990749    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:05:20.006245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:20.006200    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-w2rps" podStartSLOduration=66.843220917 podStartE2EDuration="1m10.006188405s" totalImagesPullingTime="3.162967488s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:05:20.005473556 +0000 UTC m=+69.729788742" watchObservedRunningTime="2026-07-23 00:05:20.006188405 +0000 UTC m=+69.730503591"
Jul 23 00:05:21.997825 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:21.997786    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-v79jt" event={"ID":"07bb7baf-47aa-4d60-9bf1-881afa525a35","Type":"ContainerStarted","Data":"5a2a9f2385de30810de9fb4aa3ba15a1642ebe3fb797302eff41528c0cd8f88d"}
Jul 23 00:05:22.013957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:22.013907    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-v79jt" podStartSLOduration=66.657022533 podStartE2EDuration="1m11.013890794s" totalImagesPullingTime="4.356868261s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:04:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:05:22.013264837 +0000 UTC m=+71.737580020" watchObservedRunningTime="2026-07-23 00:05:22.013890794 +0000 UTC m=+71.738205978"
Jul 23 00:05:48.994688 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:48.994650    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:05:48.995174 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:48.994694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:05:48.995174 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:48.994801    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:05:48.995174 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:48.994817    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:05:48.995174 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:48.994828    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:05:48.995174 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:48.994917    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:06:52.994901201 +0000 UTC m=+162.719216363 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:05:48.995174 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:05:48.994932    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:06:52.994925879 +0000 UTC m=+162.719241041 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:05:50.996230 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:05:50.996204    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-w2rps"
Jul 23 00:06:20.402988 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:06:20.402941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:06:20.403490 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:20.403069    2558 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 23 00:06:20.403490 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:20.403144    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs podName:dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a nodeName:}" failed. No retries permitted until 2026-07-23 00:08:22.403124962 +0000 UTC m=+252.127440142 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs") pod "network-metrics-daemon-csrhk" (UID: "dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a") : secret "metrics-daemon-secret" not found
Jul 23 00:06:48.148292 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:48.148241    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa"
Jul 23 00:06:48.183433 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:48.183388    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ttkgf" podUID="23918e6b-3a8a-4f61-98c5-dc0f21cb1b23"
Jul 23 00:06:48.204435 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:06:48.204405    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:06:48.204542 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:06:48.204439    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:06:49.740152 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:49.740103    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-csrhk" podUID="dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a"
Jul 23 00:06:53.015000 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:06:53.014955    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:06:53.015430 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:06:53.015027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:06:53.015430 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:53.015112    2558 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 23 00:06:53.015430 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:53.015136    2558 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7cfbc886f-76f7c: secret "image-registry-tls" not found
Jul 23 00:06:53.015430 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:53.015120    2558 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 23 00:06:53.015430 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:53.015226    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls podName:55f33850-7ffa-4ed1-83d7-477a646618fa nodeName:}" failed. No retries permitted until 2026-07-23 00:08:55.015208093 +0000 UTC m=+284.739523259 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls") pod "image-registry-7cfbc886f-76f7c" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa") : secret "image-registry-tls" not found
Jul 23 00:06:53.015430 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:06:53.015262    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls podName:23918e6b-3a8a-4f61-98c5-dc0f21cb1b23 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:55.015246842 +0000 UTC m=+284.739562007 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls") pod "dns-default-ttkgf" (UID: "23918e6b-3a8a-4f61-98c5-dc0f21cb1b23") : secret "dns-default-metrics-tls" not found
Jul 23 00:07:03.728761 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:07:03.728675    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:08:04.162803 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.162751    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"]
Jul 23 00:08:04.168226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.168197    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"]
Jul 23 00:08:04.168371 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.168300    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"
Jul 23 00:08:04.170405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.170376    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-v29f8\""
Jul 23 00:08:04.171090 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.171067    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.172996 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.172979    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 23 00:08:04.174580 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.174557    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9jvws\""
Jul 23 00:08:04.174705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.174652    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 23 00:08:04.174705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.174655    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 23 00:08:04.174886 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.174861    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 23 00:08:04.176011 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.175652    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"]
Jul 23 00:08:04.186149 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.186116    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"]
Jul 23 00:08:04.259466 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.259437    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-p5kxt"]
Jul 23 00:08:04.262205 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.262187    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-56d94dd6b-z49t2"]
Jul 23 00:08:04.262330 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.262315    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.264734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.264719    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 23 00:08:04.264998 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.264979    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.265191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.265062    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 23 00:08:04.265191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.265182    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-2grlr\""
Jul 23 00:08:04.265381 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.265362    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 23 00:08:04.265594 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.265579    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 23 00:08:04.270200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270182    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 23 00:08:04.270278 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270198    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 23 00:08:04.270456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270441    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 23 00:08:04.270539 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270524    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 23 00:08:04.270595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270537    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-274h7\""
Jul 23 00:08:04.270595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270540    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 23 00:08:04.270595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.270594    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 23 00:08:04.271634 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.271614    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 23 00:08:04.276748 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.276727    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-p5kxt"]
Jul 23 00:08:04.279979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.279955    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56d94dd6b-z49t2"]
Jul 23 00:08:04.292099 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.292080    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.292186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.292106    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xmjv\" (UniqueName: \"kubernetes.io/projected/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-kube-api-access-5xmjv\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.292186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.292148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpgc9\" (UniqueName: \"kubernetes.io/projected/fcd43717-7ff2-498e-b47f-0acb99604256-kube-api-access-hpgc9\") pod \"network-check-source-7bc5b6bf85-kcrvs\" (UID: \"fcd43717-7ff2-498e-b47f-0acb99604256\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"
Jul 23 00:08:04.292255 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.292189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.370741 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.370710    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"]
Jul 23 00:08:04.373565 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.373550    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"
Jul 23 00:08:04.379979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.379962    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-wplx7\""
Jul 23 00:08:04.381806 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.381793    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:08:04.385812 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.385799    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 23 00:08:04.391387 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.391368    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"]
Jul 23 00:08:04.392755 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392735    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.392882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29935393-8973-47ec-9c30-b7a21f0e270f-tmp\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.392882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392797    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-default-certificate\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.392882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392815    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.392882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392863    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/29935393-8973-47ec-9c30-b7a21f0e270f-snapshots\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.393118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392888    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29935393-8973-47ec-9c30-b7a21f0e270f-serving-cert\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.393118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-stats-auth\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.393118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392965    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.393118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.392989    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpgc9\" (UniqueName: \"kubernetes.io/projected/fcd43717-7ff2-498e-b47f-0acb99604256-kube-api-access-hpgc9\") pod \"network-check-source-7bc5b6bf85-kcrvs\" (UID: \"fcd43717-7ff2-498e-b47f-0acb99604256\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"
Jul 23 00:08:04.393118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393096    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29935393-8973-47ec-9c30-b7a21f0e270f-service-ca-bundle\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.393365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj58b\" (UniqueName: \"kubernetes.io/projected/29935393-8973-47ec-9c30-b7a21f0e270f-kube-api-access-kj58b\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.393365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393220    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wdtb\" (UniqueName: \"kubernetes.io/projected/bddc0a11-3d3b-4b42-90a0-a2715c845942-kube-api-access-6wdtb\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.393365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393251    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xmjv\" (UniqueName: \"kubernetes.io/projected/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-kube-api-access-5xmjv\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.393365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.393365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393305    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29935393-8973-47ec-9c30-b7a21f0e270f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.393607 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.393421    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:04.393607 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.393511    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls podName:ca6d1813-9908-48e9-9c00-1cb34c39b3c9 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:04.893494779 +0000 UTC m=+234.617809955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jdss5" (UID: "ca6d1813-9908-48e9-9c00-1cb34c39b3c9") : secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:04.393607 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.393543    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.402225 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.402207    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpgc9\" (UniqueName: \"kubernetes.io/projected/fcd43717-7ff2-498e-b47f-0acb99604256-kube-api-access-hpgc9\") pod \"network-check-source-7bc5b6bf85-kcrvs\" (UID: \"fcd43717-7ff2-498e-b47f-0acb99604256\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"
Jul 23 00:08:04.403787 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.403763    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xmjv\" (UniqueName: \"kubernetes.io/projected/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-kube-api-access-5xmjv\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.479648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.479599    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"
Jul 23 00:08:04.494603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494583    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29935393-8973-47ec-9c30-b7a21f0e270f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.494713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494613    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29935393-8973-47ec-9c30-b7a21f0e270f-tmp\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.494713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494636    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-default-certificate\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.494713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494654    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.494959 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.494760    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 00:08:04.494959 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494775    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/29935393-8973-47ec-9c30-b7a21f0e270f-snapshots\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.494959 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.494815    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:04.994797372 +0000 UTC m=+234.719112533 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : secret "router-metrics-certs-default" not found
Jul 23 00:08:04.494959 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494860    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29935393-8973-47ec-9c30-b7a21f0e270f-serving-cert\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.494959 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494901    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-stats-auth\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.494959 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494944    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.495314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.494979    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbmxg\" (UniqueName: \"kubernetes.io/projected/65b77b76-07fb-4d85-9a53-a9e91029aae0-kube-api-access-fbmxg\") pod \"volume-data-source-validator-78cf4c6f6-9nkpn\" (UID: \"65b77b76-07fb-4d85-9a53-a9e91029aae0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"
Jul 23 00:08:04.495314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495015    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29935393-8973-47ec-9c30-b7a21f0e270f-service-ca-bundle\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.495314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495051    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj58b\" (UniqueName: \"kubernetes.io/projected/29935393-8973-47ec-9c30-b7a21f0e270f-kube-api-access-kj58b\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.495314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495059    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/29935393-8973-47ec-9c30-b7a21f0e270f-tmp\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.495314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495087    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wdtb\" (UniqueName: \"kubernetes.io/projected/bddc0a11-3d3b-4b42-90a0-a2715c845942-kube-api-access-6wdtb\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.495314 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.495175    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:04.9951566 +0000 UTC m=+234.719471779 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : configmap references non-existent config key: service-ca.crt
Jul 23 00:08:04.495705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495682    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29935393-8973-47ec-9c30-b7a21f0e270f-service-ca-bundle\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.495773 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495701    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/29935393-8973-47ec-9c30-b7a21f0e270f-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.495909 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.495892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/29935393-8973-47ec-9c30-b7a21f0e270f-snapshots\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.497255 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.497235    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/29935393-8973-47ec-9c30-b7a21f0e270f-serving-cert\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.497520 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.497503    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-default-certificate\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.497584 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.497527    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-stats-auth\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.503853 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.503806    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj58b\" (UniqueName: \"kubernetes.io/projected/29935393-8973-47ec-9c30-b7a21f0e270f-kube-api-access-kj58b\") pod \"insights-operator-555d9d7885-p5kxt\" (UID: \"29935393-8973-47ec-9c30-b7a21f0e270f\") " pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.504284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.504264    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wdtb\" (UniqueName: \"kubernetes.io/projected/bddc0a11-3d3b-4b42-90a0-a2715c845942-kube-api-access-6wdtb\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.572400 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.572371    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-p5kxt"
Jul 23 00:08:04.595909 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.595880    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbmxg\" (UniqueName: \"kubernetes.io/projected/65b77b76-07fb-4d85-9a53-a9e91029aae0-kube-api-access-fbmxg\") pod \"volume-data-source-validator-78cf4c6f6-9nkpn\" (UID: \"65b77b76-07fb-4d85-9a53-a9e91029aae0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"
Jul 23 00:08:04.598480 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.598458    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs"]
Jul 23 00:08:04.601208 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:04.601183    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcd43717_7ff2_498e_b47f_0acb99604256.slice/crio-0e76238402a822bb318585dc14bae4fae0354ec43b7df17b2d32c2aaf1bad12c WatchSource:0}: Error finding container 0e76238402a822bb318585dc14bae4fae0354ec43b7df17b2d32c2aaf1bad12c: Status 404 returned error can't find the container with id 0e76238402a822bb318585dc14bae4fae0354ec43b7df17b2d32c2aaf1bad12c
Jul 23 00:08:04.605329 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.605310    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbmxg\" (UniqueName: \"kubernetes.io/projected/65b77b76-07fb-4d85-9a53-a9e91029aae0-kube-api-access-fbmxg\") pod \"volume-data-source-validator-78cf4c6f6-9nkpn\" (UID: \"65b77b76-07fb-4d85-9a53-a9e91029aae0\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"
Jul 23 00:08:04.681937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.681911    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"
Jul 23 00:08:04.695363 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.695332    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-p5kxt"]
Jul 23 00:08:04.697711 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:04.697689    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29935393_8973_47ec_9c30_b7a21f0e270f.slice/crio-6b69eabb3f750404eaead69ff20b7efeccdd9f3bd17881ea7c877a3e6b21b649 WatchSource:0}: Error finding container 6b69eabb3f750404eaead69ff20b7efeccdd9f3bd17881ea7c877a3e6b21b649: Status 404 returned error can't find the container with id 6b69eabb3f750404eaead69ff20b7efeccdd9f3bd17881ea7c877a3e6b21b649
Jul 23 00:08:04.799673 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.799644    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn"]
Jul 23 00:08:04.802073 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:04.802049    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65b77b76_07fb_4d85_9a53_a9e91029aae0.slice/crio-dddb48bccf361d2ad6b7b4f19b5da4fa794e17405007069a4c394ff08c923e80 WatchSource:0}: Error finding container dddb48bccf361d2ad6b7b4f19b5da4fa794e17405007069a4c394ff08c923e80: Status 404 returned error can't find the container with id dddb48bccf361d2ad6b7b4f19b5da4fa794e17405007069a4c394ff08c923e80
Jul 23 00:08:04.897886 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.897864    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:04.898019 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.898003    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:04.898071 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.898057    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls podName:ca6d1813-9908-48e9-9c00-1cb34c39b3c9 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:05.898042094 +0000 UTC m=+235.622357256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jdss5" (UID: "ca6d1813-9908-48e9-9c00-1cb34c39b3c9") : secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:04.999009 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.998984    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.999088 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:04.999028    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:04.999145 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.999108    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:05.999096797 +0000 UTC m=+235.723411959 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : configmap references non-existent config key: service-ca.crt
Jul 23 00:08:04.999145 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.999108    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 00:08:04.999145 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:04.999138    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:05.999128605 +0000 UTC m=+235.723443768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : secret "router-metrics-certs-default" not found
Jul 23 00:08:05.405687 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:05.405558    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn" event={"ID":"65b77b76-07fb-4d85-9a53-a9e91029aae0","Type":"ContainerStarted","Data":"dddb48bccf361d2ad6b7b4f19b5da4fa794e17405007069a4c394ff08c923e80"}
Jul 23 00:08:05.408157 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:05.408082    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-p5kxt" event={"ID":"29935393-8973-47ec-9c30-b7a21f0e270f","Type":"ContainerStarted","Data":"6b69eabb3f750404eaead69ff20b7efeccdd9f3bd17881ea7c877a3e6b21b649"}
Jul 23 00:08:05.411517 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:05.411482    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs" event={"ID":"fcd43717-7ff2-498e-b47f-0acb99604256","Type":"ContainerStarted","Data":"b400d0c893ead0cf72228bf4e26e77d9d29c16ef11fbe47e91a72b6345c18cef"}
Jul 23 00:08:05.411636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:05.411524    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs" event={"ID":"fcd43717-7ff2-498e-b47f-0acb99604256","Type":"ContainerStarted","Data":"0e76238402a822bb318585dc14bae4fae0354ec43b7df17b2d32c2aaf1bad12c"}
Jul 23 00:08:05.429851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:05.429788    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-kcrvs" podStartSLOduration=1.429772817 podStartE2EDuration="1.429772817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:05.429559094 +0000 UTC m=+235.153874280" watchObservedRunningTime="2026-07-23 00:08:05.429772817 +0000 UTC m=+235.154088002"
Jul 23 00:08:05.905423 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:05.905398    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:05.905564 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:05.905544    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:05.905630 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:05.905617    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls podName:ca6d1813-9908-48e9-9c00-1cb34c39b3c9 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:07.905596137 +0000 UTC m=+237.629911312 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jdss5" (UID: "ca6d1813-9908-48e9-9c00-1cb34c39b3c9") : secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:06.005975 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:06.005929    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:06.006116 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:06.006017    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:06.006116 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:06.006055    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 00:08:06.006237 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:06.006119    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:08.006099675 +0000 UTC m=+237.730414839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : secret "router-metrics-certs-default" not found
Jul 23 00:08:06.006237 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:06.006164    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:08.006151051 +0000 UTC m=+237.730466223 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : configmap references non-existent config key: service-ca.crt
Jul 23 00:08:06.415995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:06.415954    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn" event={"ID":"65b77b76-07fb-4d85-9a53-a9e91029aae0","Type":"ContainerStarted","Data":"5a5afb505cdee535eb570c41f67d8fa8077cb90d1e85bb3f314d9b83a7045daa"}
Jul 23 00:08:06.432592 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:06.432541    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-9nkpn" podStartSLOduration=1.334565956 podStartE2EDuration="2.432521957s" totalImagesPullingTime="1.097956001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:06.430865036 +0000 UTC m=+236.155180219" watchObservedRunningTime="2026-07-23 00:08:06.432521957 +0000 UTC m=+236.156837144"
Jul 23 00:08:07.420410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:07.420373    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-p5kxt" event={"ID":"29935393-8973-47ec-9c30-b7a21f0e270f","Type":"ContainerStarted","Data":"277a41f1241963eb33dab166079cad79e31dbd933973405da9ee1dbba4782b1f"}
Jul 23 00:08:07.438011 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:07.437967    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-p5kxt" podStartSLOduration=1.420044459 podStartE2EDuration="3.437951554s" totalImagesPullingTime="2.017907095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:07.437042221 +0000 UTC m=+237.161357423" watchObservedRunningTime="2026-07-23 00:08:07.437951554 +0000 UTC m=+237.162266966"
Jul 23 00:08:07.920659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:07.920629    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:07.920798 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:07.920780    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:07.920877 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:07.920870    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls podName:ca6d1813-9908-48e9-9c00-1cb34c39b3c9 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:11.920850392 +0000 UTC m=+241.645165569 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jdss5" (UID: "ca6d1813-9908-48e9-9c00-1cb34c39b3c9") : secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:08.021490 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.021462    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:08.021622 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.021521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:08.021622 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.021589    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 00:08:08.021696 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.021650    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:12.021633317 +0000 UTC m=+241.745948489 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : secret "router-metrics-certs-default" not found
Jul 23 00:08:08.021696 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.021678    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:12.021663196 +0000 UTC m=+241.745978358 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : configmap references non-existent config key: service-ca.crt
Jul 23 00:08:08.173831 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.173751    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-csg5h"]
Jul 23 00:08:08.177399 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.177378    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.182487 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.182280    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 23 00:08:08.182487 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.182351    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7jhfv\""
Jul 23 00:08:08.182487 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.182435    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 23 00:08:08.196345 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.196324    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-csg5h"]
Jul 23 00:08:08.224107 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.224076    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.224214 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.224114    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.224214 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.224159    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7902e54a-d09e-446d-b377-3ddf56e3624e-crio-socket\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.224342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.224319    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7902e54a-d09e-446d-b377-3ddf56e3624e-data-volume\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.224374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.224353    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8ctb\" (UniqueName: \"kubernetes.io/projected/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-api-access-w8ctb\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.325664 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325641    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7902e54a-d09e-446d-b377-3ddf56e3624e-data-volume\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.325664 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8ctb\" (UniqueName: \"kubernetes.io/projected/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-api-access-w8ctb\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.325825 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325688    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.325825 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325708    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.325825 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.325787    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:08.325825 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.325791    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:08:08.326000 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.325867    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:08.825827927 +0000 UTC m=+238.550143091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:08.326000 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325895    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7902e54a-d09e-446d-b377-3ddf56e3624e-crio-socket\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.326000 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.325949    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:08.825928079 +0000 UTC m=+238.550243254 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : configmap "kube-rbac-proxy" not found
Jul 23 00:08:08.326000 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325958    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/7902e54a-d09e-446d-b377-3ddf56e3624e-data-volume\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.326000 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.325972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/7902e54a-d09e-446d-b377-3ddf56e3624e-crio-socket\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.333270 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.333243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8ctb\" (UniqueName: \"kubernetes.io/projected/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-api-access-w8ctb\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.830148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.830110    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.830148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:08.830153    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:08.830545 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.830242    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:08.830545 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.830244    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:08:08.830545 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.830319    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:09.830306606 +0000 UTC m=+239.554621769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:08.830545 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:08.830333    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:09.830326541 +0000 UTC m=+239.554641703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : configmap "kube-rbac-proxy" not found
Jul 23 00:08:09.836713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:09.836685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:09.836713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:09.836715    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:09.837291 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:09.836791    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:08:09.837291 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:09.836811    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:09.837291 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:09.836873    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:11.836860819 +0000 UTC m=+241.561175981 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:09.837291 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:09.836889    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:11.836881685 +0000 UTC m=+241.561196847 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : configmap "kube-rbac-proxy" not found
Jul 23 00:08:10.366140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:10.366114    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fcgjk_7509dbfc-f6e2-46ea-a785-32e381851a89/dns-node-resolver/0.log"
Jul 23 00:08:10.570544 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:10.570514    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29746080-c9kg2_1b718f23-82f5-4b53-9f6b-e17f42514a82/image-pruner/0.log"
Jul 23 00:08:11.114753 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.114718    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-27lnh"]
Jul 23 00:08:11.118019 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.118004    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.120535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.120510    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-wpr5n\""
Jul 23 00:08:11.120535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.120528    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 23 00:08:11.121334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.121320    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 23 00:08:11.121394 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.121341    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 23 00:08:11.121438 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.121417    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 23 00:08:11.125451 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.125430    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-27lnh"]
Jul 23 00:08:11.165158 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.165139    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5sfgf_78c775d2-bca1-4090-837a-078c4a09fb98/node-ca/0.log"
Jul 23 00:08:11.248628 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.248600    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-signing-key\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.248748 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.248673    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-signing-cabundle\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.248748 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.248722    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r2dk\" (UniqueName: \"kubernetes.io/projected/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-kube-api-access-8r2dk\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.349534 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.349506    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-signing-key\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.349648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.349562    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-signing-cabundle\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.349648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.349600    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r2dk\" (UniqueName: \"kubernetes.io/projected/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-kube-api-access-8r2dk\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.350315 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.350292    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-signing-cabundle\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.351908 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.351889    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-signing-key\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.358967 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.358944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r2dk\" (UniqueName: \"kubernetes.io/projected/3a1c78f2-b40f-4a0a-a88b-f172eb724b42-kube-api-access-8r2dk\") pod \"service-ca-58598695f7-27lnh\" (UID: \"3a1c78f2-b40f-4a0a-a88b-f172eb724b42\") " pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.427907 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.427860    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-27lnh"
Jul 23 00:08:11.545302 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.545280    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-27lnh"]
Jul 23 00:08:11.547347 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:11.547323    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1c78f2_b40f_4a0a_a88b_f172eb724b42.slice/crio-5e1e7d72f4a1eaa0ae2728446b57d2e92384136fb7939563255015ea8b6e108f WatchSource:0}: Error finding container 5e1e7d72f4a1eaa0ae2728446b57d2e92384136fb7939563255015ea8b6e108f: Status 404 returned error can't find the container with id 5e1e7d72f4a1eaa0ae2728446b57d2e92384136fb7939563255015ea8b6e108f
Jul 23 00:08:11.853325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.853300    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:11.853444 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.853335    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:11.853444 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:11.853430    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:08:11.853519 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:11.853467    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:11.853519 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:11.853494    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:15.853475908 +0000 UTC m=+245.577791084 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : configmap "kube-rbac-proxy" not found
Jul 23 00:08:11.853586 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:11.853528    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:15.853516838 +0000 UTC m=+245.577832001 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:11.954456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:11.954431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:11.954571 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:11.954523    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:11.954571 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:11.954567    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls podName:ca6d1813-9908-48e9-9c00-1cb34c39b3c9 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:19.954555147 +0000 UTC m=+249.678870312 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jdss5" (UID: "ca6d1813-9908-48e9-9c00-1cb34c39b3c9") : secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:12.055494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:12.055460    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:12.055599 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:12.055536    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:12.055599 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:12.055570    2558 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 23 00:08:12.055698 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:12.055649    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:20.055632752 +0000 UTC m=+249.779947918 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : secret "router-metrics-certs-default" not found
Jul 23 00:08:12.055766 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:12.055708    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:20.055691451 +0000 UTC m=+249.780006617 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : configmap references non-existent config key: service-ca.crt
Jul 23 00:08:12.165983 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:12.165929    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l995d_5a60f4c3-d54a-42ef-93ef-899e4e819daf/serve-healthcheck-canary/0.log"
Jul 23 00:08:12.437015 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:12.436946    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-27lnh" event={"ID":"3a1c78f2-b40f-4a0a-a88b-f172eb724b42","Type":"ContainerStarted","Data":"5e1e7d72f4a1eaa0ae2728446b57d2e92384136fb7939563255015ea8b6e108f"}
Jul 23 00:08:13.441122 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:13.441087    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-27lnh" event={"ID":"3a1c78f2-b40f-4a0a-a88b-f172eb724b42","Type":"ContainerStarted","Data":"f75a1fadfda64d0e5976189c4e727f3a24afbe0d305f89a0786ec223d19d669d"}
Jul 23 00:08:13.456980 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:13.456933    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-27lnh" podStartSLOduration=0.6597752 podStartE2EDuration="2.456918257s" totalImagesPullingTime="1.797143057s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:13.456418656 +0000 UTC m=+243.180733840" watchObservedRunningTime="2026-07-23 00:08:13.456918257 +0000 UTC m=+243.181233440"
Jul 23 00:08:15.886591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:15.886557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:15.886591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:15.886591    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:15.887005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:15.886666    2558 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 23 00:08:15.887005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:15.886683    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:15.887005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:15.886729    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:23.886711906 +0000 UTC m=+253.611027080 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:15.887005 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:15.886796    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:23.886787444 +0000 UTC m=+253.611102611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : configmap "kube-rbac-proxy" not found
Jul 23 00:08:20.018595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:20.018558    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:20.019066 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:20.018728    2558 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:20.019066 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:20.018826    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls podName:ca6d1813-9908-48e9-9c00-1cb34c39b3c9 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:36.018805709 +0000 UTC m=+265.743120885 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-jdss5" (UID: "ca6d1813-9908-48e9-9c00-1cb34c39b3c9") : secret "cluster-monitoring-operator-tls" not found
Jul 23 00:08:20.119473 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:20.119446    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:20.119612 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:20.119595    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:20.119763 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:20.119746    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle podName:bddc0a11-3d3b-4b42-90a0-a2715c845942 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:36.119727629 +0000 UTC m=+265.844042796 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle") pod "router-default-56d94dd6b-z49t2" (UID: "bddc0a11-3d3b-4b42-90a0-a2715c845942") : configmap references non-existent config key: service-ca.crt
Jul 23 00:08:20.121938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:20.121922    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bddc0a11-3d3b-4b42-90a0-a2715c845942-metrics-certs\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:22.437372 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:22.437334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:08:22.440062 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:22.440039    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a-metrics-certs\") pod \"network-metrics-daemon-csrhk\" (UID: \"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a\") " pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:08:22.631758 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:22.631730    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-djfbd\""
Jul 23 00:08:22.640724 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:22.640705    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-csrhk"
Jul 23 00:08:22.761820 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:22.761791    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-csrhk"]
Jul 23 00:08:22.764867 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:22.764826    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddcc00c5a_19e5_44ec_bfc8_84c6542c0a4a.slice/crio-32c7c8897d928d629dca31fa7220edf151638a41c0fc59af821db0288e3fdb72 WatchSource:0}: Error finding container 32c7c8897d928d629dca31fa7220edf151638a41c0fc59af821db0288e3fdb72: Status 404 returned error can't find the container with id 32c7c8897d928d629dca31fa7220edf151638a41c0fc59af821db0288e3fdb72
Jul 23 00:08:23.474792 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:23.474760    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-csrhk" event={"ID":"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a","Type":"ContainerStarted","Data":"32c7c8897d928d629dca31fa7220edf151638a41c0fc59af821db0288e3fdb72"}
Jul 23 00:08:23.949740 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:23.949713    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:23.949875 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:23.949762    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:23.949951 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:23.949889    2558 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:23.950004 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:23.949992    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls podName:7902e54a-d09e-446d-b377-3ddf56e3624e nodeName:}" failed. No retries permitted until 2026-07-23 00:08:39.949968147 +0000 UTC m=+269.674283324 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-csg5h" (UID: "7902e54a-d09e-446d-b377-3ddf56e3624e") : secret "insights-runtime-extractor-tls" not found
Jul 23 00:08:23.950201 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:23.950171    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/7902e54a-d09e-446d-b377-3ddf56e3624e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:24.479924 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:24.479894    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-csrhk" event={"ID":"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a","Type":"ContainerStarted","Data":"e2dc169a4e71c9d3cc888eff0e6d6da117fe3e21ac2223796d1e24787b74295f"}
Jul 23 00:08:24.480256 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:24.479943    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-csrhk" event={"ID":"dcc00c5a-19e5-44ec-bfc8-84c6542c0a4a","Type":"ContainerStarted","Data":"9ec2a4b2cef68471fe3f98e3e2e6140f2a5fad4a6a8c5ebe5d1f8844dbf0f948"}
Jul 23 00:08:24.496276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:24.496231    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-csrhk" podStartSLOduration=253.37748349 podStartE2EDuration="4m14.49621644s" totalImagesPullingTime="1.11873295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:04:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:24.494702592 +0000 UTC m=+254.219017786" watchObservedRunningTime="2026-07-23 00:08:24.49621644 +0000 UTC m=+254.220531623"
Jul 23 00:08:36.040854 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.040723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:36.043511 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.043478    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/ca6d1813-9908-48e9-9c00-1cb34c39b3c9-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-jdss5\" (UID: \"ca6d1813-9908-48e9-9c00-1cb34c39b3c9\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:36.141870 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.141820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:36.142398 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.142373    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bddc0a11-3d3b-4b42-90a0-a2715c845942-service-ca-bundle\") pod \"router-default-56d94dd6b-z49t2\" (UID: \"bddc0a11-3d3b-4b42-90a0-a2715c845942\") " pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:36.286978 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.286952    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-9jvws\""
Jul 23 00:08:36.294821 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.294770    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"
Jul 23 00:08:36.379326 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.379302    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-274h7\""
Jul 23 00:08:36.387241 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.387214    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:36.424765 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.424740    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5"]
Jul 23 00:08:36.428799 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:36.428768    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca6d1813_9908_48e9_9c00_1cb34c39b3c9.slice/crio-a8b8e58586d42ec6560899f90e1a6220e484582f080b61c3cc2d32829ebc73be WatchSource:0}: Error finding container a8b8e58586d42ec6560899f90e1a6220e484582f080b61c3cc2d32829ebc73be: Status 404 returned error can't find the container with id a8b8e58586d42ec6560899f90e1a6220e484582f080b61c3cc2d32829ebc73be
Jul 23 00:08:36.511297 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.511206    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-56d94dd6b-z49t2"]
Jul 23 00:08:36.513710 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:36.513687    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbddc0a11_3d3b_4b42_90a0_a2715c845942.slice/crio-ac9e2cae5fe8809735c1608dbc63272971a5b5aacf632b6530464212ee686cd3 WatchSource:0}: Error finding container ac9e2cae5fe8809735c1608dbc63272971a5b5aacf632b6530464212ee686cd3: Status 404 returned error can't find the container with id ac9e2cae5fe8809735c1608dbc63272971a5b5aacf632b6530464212ee686cd3
Jul 23 00:08:36.523745 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.523718    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56d94dd6b-z49t2" event={"ID":"bddc0a11-3d3b-4b42-90a0-a2715c845942","Type":"ContainerStarted","Data":"ac9e2cae5fe8809735c1608dbc63272971a5b5aacf632b6530464212ee686cd3"}
Jul 23 00:08:36.525004 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:36.524979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5" event={"ID":"ca6d1813-9908-48e9-9c00-1cb34c39b3c9","Type":"ContainerStarted","Data":"a8b8e58586d42ec6560899f90e1a6220e484582f080b61c3cc2d32829ebc73be"}
Jul 23 00:08:37.529179 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:37.529142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-56d94dd6b-z49t2" event={"ID":"bddc0a11-3d3b-4b42-90a0-a2715c845942","Type":"ContainerStarted","Data":"83cc288b09810fc498b9b0c72f6a415e709ad1351c8cf435e15743a6e35842af"}
Jul 23 00:08:37.550494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:37.550449    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-56d94dd6b-z49t2" podStartSLOduration=33.550428112 podStartE2EDuration="33.550428112s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:37.548990982 +0000 UTC m=+267.273306204" watchObservedRunningTime="2026-07-23 00:08:37.550428112 +0000 UTC m=+267.274743297"
Jul 23 00:08:38.388250 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:38.388214    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:38.391118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:38.391092    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:38.532062 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:38.532032    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:38.533259 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:38.533237    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-56d94dd6b-z49t2"
Jul 23 00:08:39.040489 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.040456    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"]
Jul 23 00:08:39.043563 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.043540    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:39.045953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.045933    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 23 00:08:39.046053 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.045963    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-sfbxc\""
Jul 23 00:08:39.054933 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.054910    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"]
Jul 23 00:08:39.167022 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.167001    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab615151-090f-437f-a794-839373039819-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pt7bw\" (UID: \"ab615151-090f-437f-a794-839373039819\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:39.268248 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.268226    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab615151-090f-437f-a794-839373039819-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pt7bw\" (UID: \"ab615151-090f-437f-a794-839373039819\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:39.270428 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.270410    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ab615151-090f-437f-a794-839373039819-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-pt7bw\" (UID: \"ab615151-090f-437f-a794-839373039819\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:39.352443 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.352412    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:39.468122 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.468101    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"]
Jul 23 00:08:39.470526 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:39.470504    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab615151_090f_437f_a794_839373039819.slice/crio-ba9e20343cfcb30b43a1481c3eb205f16c94aa7e4e5ab13a7b4392b1a8c8ab82 WatchSource:0}: Error finding container ba9e20343cfcb30b43a1481c3eb205f16c94aa7e4e5ab13a7b4392b1a8c8ab82: Status 404 returned error can't find the container with id ba9e20343cfcb30b43a1481c3eb205f16c94aa7e4e5ab13a7b4392b1a8c8ab82
Jul 23 00:08:39.535414 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.535386    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw" event={"ID":"ab615151-090f-437f-a794-839373039819","Type":"ContainerStarted","Data":"ba9e20343cfcb30b43a1481c3eb205f16c94aa7e4e5ab13a7b4392b1a8c8ab82"}
Jul 23 00:08:39.536693 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.536672    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5" event={"ID":"ca6d1813-9908-48e9-9c00-1cb34c39b3c9","Type":"ContainerStarted","Data":"88ace729175deaecd195a1228e17ef1d8487ab1fa337cb2474142b17263993d3"}
Jul 23 00:08:39.552242 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.552202    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-jdss5" podStartSLOduration=33.414832796 podStartE2EDuration="35.55218956s" totalImagesPullingTime="2.137356764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:39.550853368 +0000 UTC m=+269.275168543" watchObservedRunningTime="2026-07-23 00:08:39.55218956 +0000 UTC m=+269.276504744"
Jul 23 00:08:39.975344 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.975318    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:39.977566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.977542    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/7902e54a-d09e-446d-b377-3ddf56e3624e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-csg5h\" (UID: \"7902e54a-d09e-446d-b377-3ddf56e3624e\") " pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:39.988766 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.988746    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-7jhfv\""
Jul 23 00:08:39.996658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:39.996643    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-csg5h"
Jul 23 00:08:40.139518 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:40.139491    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-csg5h"]
Jul 23 00:08:40.141658 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:40.141634    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7902e54a_d09e_446d_b377_3ddf56e3624e.slice/crio-d4431e22562d224a9ec63e582e35aab59356c07376216d57ce976db7a6b9e7b6 WatchSource:0}: Error finding container d4431e22562d224a9ec63e582e35aab59356c07376216d57ce976db7a6b9e7b6: Status 404 returned error can't find the container with id d4431e22562d224a9ec63e582e35aab59356c07376216d57ce976db7a6b9e7b6
Jul 23 00:08:40.540678 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:40.540641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csg5h" event={"ID":"7902e54a-d09e-446d-b377-3ddf56e3624e","Type":"ContainerStarted","Data":"d4431e22562d224a9ec63e582e35aab59356c07376216d57ce976db7a6b9e7b6"}
Jul 23 00:08:41.545370 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:41.545284    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw" event={"ID":"ab615151-090f-437f-a794-839373039819","Type":"ContainerStarted","Data":"78bda2e895a9be1e2cf4e7feb66566ce2d105728b34bee0c5d57bbf82d7068ef"}
Jul 23 00:08:41.545777 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:41.545504    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:41.550988 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:41.550963    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw"
Jul 23 00:08:41.560431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:41.560393    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-pt7bw" podStartSLOduration=0.814070271 podStartE2EDuration="2.560379315s" totalImagesPullingTime="1.746309044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:41.559498269 +0000 UTC m=+271.283813453" watchObservedRunningTime="2026-07-23 00:08:41.560379315 +0000 UTC m=+271.284694500"
Jul 23 00:08:45.562145 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:45.562103    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csg5h" event={"ID":"7902e54a-d09e-446d-b377-3ddf56e3624e","Type":"ContainerStarted","Data":"2ed2a04339ac90a3a16f71c69e3e440f42eeb346bbaf4c1b82179ab3dd9dc52d"}
Jul 23 00:08:48.572875 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:48.572831    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csg5h" event={"ID":"7902e54a-d09e-446d-b377-3ddf56e3624e","Type":"ContainerStarted","Data":"cb3e715f9178f3dd8aeeb31d8ab81d0248e3536485cc826125a8872c40dac3e0"}
Jul 23 00:08:51.210340 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:51.206202    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa"
Jul 23 00:08:51.210340 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:51.206202    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ttkgf" podUID="23918e6b-3a8a-4f61-98c5-dc0f21cb1b23"
Jul 23 00:08:51.441132 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.441086    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"]
Jul 23 00:08:51.469636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.469541    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"]
Jul 23 00:08:51.469636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.469596    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2w2sb"]
Jul 23 00:08:51.469636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.469632    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.473732 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.472968    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 23 00:08:51.473732 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.473187    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-v5b7q\""
Jul 23 00:08:51.473732 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.473431    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 23 00:08:51.473732 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.473596    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 23 00:08:51.475117 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.474730    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.476892 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.476715    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 23 00:08:51.479069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.478724    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 23 00:08:51.479069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.478888    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 23 00:08:51.479069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.478988    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-fmrvj\""
Jul 23 00:08:51.567594 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567559    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.567743 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567615    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4npd\" (UniqueName: \"kubernetes.io/projected/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-kube-api-access-g4npd\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.567743 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567685    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.567743 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567715    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.567923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567745    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.567923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567771    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-sys\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.567923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567798    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.567923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567903    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-textfile\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.568121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567946    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rdfjx\" (UniqueName: \"kubernetes.io/projected/265e6475-c359-4a73-b86c-905041a290bf-kube-api-access-rdfjx\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.568121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.567975    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265e6475-c359-4a73-b86c-905041a290bf-metrics-client-ca\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.568121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.568008    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.568121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.568031    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-root\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.568121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.568054    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-wtmp\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.583414 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.583379    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:08:51.583661 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.583641    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.668933    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-textfile\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669011    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rdfjx\" (UniqueName: \"kubernetes.io/projected/265e6475-c359-4a73-b86c-905041a290bf-kube-api-access-rdfjx\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669052    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265e6475-c359-4a73-b86c-905041a290bf-metrics-client-ca\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669084    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669108    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-root\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669131    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-wtmp\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669156    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669189    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4npd\" (UniqueName: \"kubernetes.io/projected/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-kube-api-access-g4npd\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669281    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669309    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669337    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669361    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-sys\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.669390    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670951 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.670422    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-textfile\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.670951 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.670735    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/265e6475-c359-4a73-b86c-905041a290bf-metrics-client-ca\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:51.671191    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:51.671260    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls podName:265e6475-c359-4a73-b86c-905041a290bf nodeName:}" failed. No retries permitted until 2026-07-23 00:08:52.171240075 +0000 UTC m=+281.895555237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls") pod "node-exporter-2w2sb" (UID: "265e6475-c359-4a73-b86c-905041a290bf") : secret "node-exporter-tls" not found
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.671300    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-sys\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.671331    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-root\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.671424    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-wtmp\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.671746    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-accelerators-collector-config\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:51.671754    2558 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 23 00:08:51.671853 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:51.671807    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-tls podName:b12e1b4d-0f8e-478a-b821-a7d01d92cb78 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:52.171792141 +0000 UTC m=+281.896107312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-8t6k2" (UID: "b12e1b4d-0f8e-478a-b821-a7d01d92cb78") : secret "openshift-state-metrics-tls" not found
Jul 23 00:08:51.672345 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.671900    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.675276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.675240    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:51.676789 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.676766    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.691470 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.691434    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4npd\" (UniqueName: \"kubernetes.io/projected/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-kube-api-access-g4npd\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:51.694175 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:51.694152    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rdfjx\" (UniqueName: \"kubernetes.io/projected/265e6475-c359-4a73-b86c-905041a290bf-kube-api-access-rdfjx\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:52.173831 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.173803    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:52.173953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.173862    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:52.173997 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:52.173954    2558 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 23 00:08:52.174030 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:52.173998    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls podName:265e6475-c359-4a73-b86c-905041a290bf nodeName:}" failed. No retries permitted until 2026-07-23 00:08:53.173983598 +0000 UTC m=+282.898298760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls") pod "node-exporter-2w2sb" (UID: "265e6475-c359-4a73-b86c-905041a290bf") : secret "node-exporter-tls" not found
Jul 23 00:08:52.175891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.175867    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/b12e1b4d-0f8e-478a-b821-a7d01d92cb78-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-8t6k2\" (UID: \"b12e1b4d-0f8e-478a-b821-a7d01d92cb78\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:52.385930 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.385909    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"
Jul 23 00:08:52.497692 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.497662    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:08:52.502431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.502399    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.504695 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.504663    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 00:08:52.504814 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.504663    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 00:08:52.504814 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.504663    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 00:08:52.504814 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.504735    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 00:08:52.505096 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.505072    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 00:08:52.505205 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.505122    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 00:08:52.505276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.505202    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 00:08:52.505276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.505076    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 00:08:52.505465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.505438    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 00:08:52.505567 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.505505    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tkvgj\""
Jul 23 00:08:52.515532 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.515513    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:08:52.519472 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.519455    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2"]
Jul 23 00:08:52.522961 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:52.522124    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb12e1b4d_0f8e_478a_b821_a7d01d92cb78.slice/crio-d014230072d03cff06bb0d26fa29666af18d33c7fe422bd09793b2553bf5c9f3 WatchSource:0}: Error finding container d014230072d03cff06bb0d26fa29666af18d33c7fe422bd09793b2553bf5c9f3: Status 404 returned error can't find the container with id d014230072d03cff06bb0d26fa29666af18d33c7fe422bd09793b2553bf5c9f3
Jul 23 00:08:52.575919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.575898    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576016 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.575931    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576016 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.575962    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576131 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576044    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576131 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576100    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576237 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576151    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-web-config\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576237 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576248    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576276    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576324    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576474 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576360    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rfdj\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-kube-api-access-8rfdj\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576474 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576401    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-config-out\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.576474 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.576428    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-config-volume\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.588138 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.588112    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2" event={"ID":"b12e1b4d-0f8e-478a-b821-a7d01d92cb78","Type":"ContainerStarted","Data":"d014230072d03cff06bb0d26fa29666af18d33c7fe422bd09793b2553bf5c9f3"}
Jul 23 00:08:52.590310 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.590291    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-csg5h" event={"ID":"7902e54a-d09e-446d-b377-3ddf56e3624e","Type":"ContainerStarted","Data":"b6b1b8a0ab61876dda8d9a7b9d98540c76514a19cb64615949d1440bc192ba6e"}
Jul 23 00:08:52.610497 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.610431    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-csg5h" podStartSLOduration=32.882348508 podStartE2EDuration="44.610417782s" totalImagesPullingTime="11.728069274s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:08:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:52.609296215 +0000 UTC m=+282.333611420" watchObservedRunningTime="2026-07-23 00:08:52.610417782 +0000 UTC m=+282.334732957"
Jul 23 00:08:52.677276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677246    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677402 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677295    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677402 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677342    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677402 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677394    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rfdj\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-kube-api-access-8rfdj\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677465    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-config-out\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677491    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-config-volume\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677524    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677545    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677575    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677616    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677647    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677689    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-web-config\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.677746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.677715    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.678705 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:52.678189    2558 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Jul 23 00:08:52.678705 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:52.678247    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls podName:74822e68-3ed8-4663-837a-aa00632b0d61 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:53.178228703 +0000 UTC m=+282.902543872 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61") : secret "alertmanager-main-tls" not found
Jul 23 00:08:52.679496 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:08:52.679119    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle podName:74822e68-3ed8-4663-837a-aa00632b0d61 nodeName:}" failed. No retries permitted until 2026-07-23 00:08:53.179099882 +0000 UTC m=+282.903415060 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61") : configmap references non-existent config key: ca-bundle.crt
Jul 23 00:08:52.680019 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.679992    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.680116 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.680038    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.680722 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.680696    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.681010 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.680987    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-config-volume\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.681105 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.681029    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-tls-assets\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.682087 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.682061    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.682171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.682139    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.683004 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.682984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-config-out\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.683166 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.683149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.683403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.683387    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-web-config\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:52.688401 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:52.688375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rfdj\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-kube-api-access-8rfdj\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:53.182682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.182651    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:53.182682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.182685    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:53.182924 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.182702    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:53.183515 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.183496    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:53.184898 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.184878    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/265e6475-c359-4a73-b86c-905041a290bf-node-exporter-tls\") pod \"node-exporter-2w2sb\" (UID: \"265e6475-c359-4a73-b86c-905041a290bf\") " pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:53.185030 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.185012    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:53.294034 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.294008    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2w2sb"
Jul 23 00:08:53.306691 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:53.306666    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod265e6475_c359_4a73_b86c_905041a290bf.slice/crio-961d6ae67154cf59ddbbda3e1a46ba471afeafee0558e336316e7883734bc36b WatchSource:0}: Error finding container 961d6ae67154cf59ddbbda3e1a46ba471afeafee0558e336316e7883734bc36b: Status 404 returned error can't find the container with id 961d6ae67154cf59ddbbda3e1a46ba471afeafee0558e336316e7883734bc36b
Jul 23 00:08:53.412362 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.412336    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:08:53.552749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.551726    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:08:53.553236 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:53.553209    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74822e68_3ed8_4663_837a_aa00632b0d61.slice/crio-c00a5b16fabc9afe7384ac0c93df17e0ad16c88806bf4a6738365d91c00a3886 WatchSource:0}: Error finding container c00a5b16fabc9afe7384ac0c93df17e0ad16c88806bf4a6738365d91c00a3886: Status 404 returned error can't find the container with id c00a5b16fabc9afe7384ac0c93df17e0ad16c88806bf4a6738365d91c00a3886
Jul 23 00:08:53.595978 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.595944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"c00a5b16fabc9afe7384ac0c93df17e0ad16c88806bf4a6738365d91c00a3886"}
Jul 23 00:08:53.598097 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.598058    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2" event={"ID":"b12e1b4d-0f8e-478a-b821-a7d01d92cb78","Type":"ContainerStarted","Data":"b1a37f1475835bd6ed0b95da676a0c8c4a1b86e4d5d02887e4f9a81d19f0d217"}
Jul 23 00:08:53.598194 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.598115    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2" event={"ID":"b12e1b4d-0f8e-478a-b821-a7d01d92cb78","Type":"ContainerStarted","Data":"620329c45df0a62095e6f6390c2f2a6749aa5d0e8c737725793d58b53f5e70a3"}
Jul 23 00:08:53.599475 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:53.599436    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2w2sb" event={"ID":"265e6475-c359-4a73-b86c-905041a290bf","Type":"ContainerStarted","Data":"961d6ae67154cf59ddbbda3e1a46ba471afeafee0558e336316e7883734bc36b"}
Jul 23 00:08:54.425085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.424942    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7885b4b854-lrjzb"]
Jul 23 00:08:54.429260 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.429240    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.432685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.432660    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 23 00:08:54.434069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.433534    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 23 00:08:54.434069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.433604    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a3925rj0bqg6s\""
Jul 23 00:08:54.434069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.433830    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 23 00:08:54.434069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.433865    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-hb6xg\""
Jul 23 00:08:54.434560 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.434538    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 23 00:08:54.434654 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.434559    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 23 00:08:54.455001 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.454977    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7885b4b854-lrjzb"]
Jul 23 00:08:54.494980 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.494932    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69584887-f0ad-4f3f-8075-fb4c9a8e6856-metrics-client-ca\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.494997    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-tls\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.495073    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-grpc-tls\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.495103    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rm6pg\" (UniqueName: \"kubernetes.io/projected/69584887-f0ad-4f3f-8075-fb4c9a8e6856-kube-api-access-rm6pg\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.495131    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.495162    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.495271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.495405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.495306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596423 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596324    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596525    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69584887-f0ad-4f3f-8075-fb4c9a8e6856-metrics-client-ca\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596574    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-tls\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596739 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596620    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-grpc-tls\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596739 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596647    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rm6pg\" (UniqueName: \"kubernetes.io/projected/69584887-f0ad-4f3f-8075-fb4c9a8e6856-kube-api-access-rm6pg\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596739 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.596739 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.596703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.597396 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.597353    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69584887-f0ad-4f3f-8075-fb4c9a8e6856-metrics-client-ca\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.599595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.599073    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.599595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.599456    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.600105 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.600072    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-grpc-tls\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.600291 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.600265    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-tls\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.600604 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.600582    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.600809 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.600789    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/69584887-f0ad-4f3f-8075-fb4c9a8e6856-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.605113 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.605092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rm6pg\" (UniqueName: \"kubernetes.io/projected/69584887-f0ad-4f3f-8075-fb4c9a8e6856-kube-api-access-rm6pg\") pod \"thanos-querier-7885b4b854-lrjzb\" (UID: \"69584887-f0ad-4f3f-8075-fb4c9a8e6856\") " pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.606193 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.606157    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2" event={"ID":"b12e1b4d-0f8e-478a-b821-a7d01d92cb78","Type":"ContainerStarted","Data":"25190cadeef05f60246cac4de96d1c58671d1aa829c8431600f61be076af148b"}
Jul 23 00:08:54.607890 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.607824    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2w2sb" event={"ID":"265e6475-c359-4a73-b86c-905041a290bf","Type":"ContainerStarted","Data":"54f6331fd288f6fa986a6ab3a5a6f4278af877e41d9bddfce5f2fcb9c35c440d"}
Jul 23 00:08:54.623378 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.623339    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-8t6k2" podStartSLOduration=2.392101636 podStartE2EDuration="3.623323787s" totalImagesPullingTime="1.231222151s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:54.621782649 +0000 UTC m=+284.346097834" watchObservedRunningTime="2026-07-23 00:08:54.623323787 +0000 UTC m=+284.347639193"
Jul 23 00:08:54.747651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.747587    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:08:54.887177 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:54.887153    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7885b4b854-lrjzb"]
Jul 23 00:08:54.888350 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:54.888329    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69584887_f0ad_4f3f_8075_fb4c9a8e6856.slice/crio-c147c155b097704b6da39212a40d1a3c434ea84dea72d803c6bf8305adbad63a WatchSource:0}: Error finding container c147c155b097704b6da39212a40d1a3c434ea84dea72d803c6bf8305adbad63a: Status 404 returned error can't find the container with id c147c155b097704b6da39212a40d1a3c434ea84dea72d803c6bf8305adbad63a
Jul 23 00:08:55.100814 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.100786    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:08:55.100969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.100826    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:08:55.103023 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.102998    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/23918e6b-3a8a-4f61-98c5-dc0f21cb1b23-metrics-tls\") pod \"dns-default-ttkgf\" (UID: \"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23\") " pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:08:55.103119 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.103051    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"image-registry-7cfbc886f-76f7c\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") " pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:08:55.189349 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.189323    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8tjsp\""
Jul 23 00:08:55.189458 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.189437    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-6vlrt\""
Jul 23 00:08:55.194286 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.194266    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:08:55.194286 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.194280    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:08:55.324967 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.324946    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7cfbc886f-76f7c"]
Jul 23 00:08:55.326812 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:55.326788    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55f33850_7ffa_4ed1_83d7_477a646618fa.slice/crio-c83b1c941cc1c60608cbd3446f1ae9fc3f7baafa038641247f88777ab5454df6 WatchSource:0}: Error finding container c83b1c941cc1c60608cbd3446f1ae9fc3f7baafa038641247f88777ab5454df6: Status 404 returned error can't find the container with id c83b1c941cc1c60608cbd3446f1ae9fc3f7baafa038641247f88777ab5454df6
Jul 23 00:08:55.338463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.338443    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ttkgf"]
Jul 23 00:08:55.340191 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:55.340167    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23918e6b_3a8a_4f61_98c5_dc0f21cb1b23.slice/crio-64d62dd985130b38a1aff880efcb7e58c0d7d9009cf94a217aaf6a132d0c7c36 WatchSource:0}: Error finding container 64d62dd985130b38a1aff880efcb7e58c0d7d9009cf94a217aaf6a132d0c7c36: Status 404 returned error can't find the container with id 64d62dd985130b38a1aff880efcb7e58c0d7d9009cf94a217aaf6a132d0c7c36
Jul 23 00:08:55.613311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.613210    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"c147c155b097704b6da39212a40d1a3c434ea84dea72d803c6bf8305adbad63a"}
Jul 23 00:08:55.615099 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.615075    2558 generic.go:356] "Generic (PLEG): container finished" podID="265e6475-c359-4a73-b86c-905041a290bf" containerID="54f6331fd288f6fa986a6ab3a5a6f4278af877e41d9bddfce5f2fcb9c35c440d" exitCode=0
Jul 23 00:08:55.615223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.615179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2w2sb" event={"ID":"265e6475-c359-4a73-b86c-905041a290bf","Type":"ContainerDied","Data":"54f6331fd288f6fa986a6ab3a5a6f4278af877e41d9bddfce5f2fcb9c35c440d"}
Jul 23 00:08:55.617005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.616978    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720" exitCode=0
Jul 23 00:08:55.617128 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.617061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"}
Jul 23 00:08:55.619769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.619702    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" event={"ID":"55f33850-7ffa-4ed1-83d7-477a646618fa","Type":"ContainerStarted","Data":"172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4"}
Jul 23 00:08:55.619769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.619742    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" event={"ID":"55f33850-7ffa-4ed1-83d7-477a646618fa","Type":"ContainerStarted","Data":"c83b1c941cc1c60608cbd3446f1ae9fc3f7baafa038641247f88777ab5454df6"}
Jul 23 00:08:55.619769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.619768    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:08:55.621484 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.621463    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ttkgf" event={"ID":"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23","Type":"ContainerStarted","Data":"64d62dd985130b38a1aff880efcb7e58c0d7d9009cf94a217aaf6a132d0c7c36"}
Jul 23 00:08:55.648640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.648591    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" podStartSLOduration=284.648576403 podStartE2EDuration="4m44.648576403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:04:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:55.646877113 +0000 UTC m=+285.371192301" watchObservedRunningTime="2026-07-23 00:08:55.648576403 +0000 UTC m=+285.372891586"
Jul 23 00:08:55.846867 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.846816    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-b76f97fd-bdsbj"]
Jul 23 00:08:55.850469 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.850441    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.852869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.852766    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 23 00:08:55.852869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.852778    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-tk8vs\""
Jul 23 00:08:55.852869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.852817    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 23 00:08:55.853166 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.852946    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 23 00:08:55.853232 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.853187    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-6as7kspua85c1\""
Jul 23 00:08:55.853335 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.853242    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 23 00:08:55.859117 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.859096    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b76f97fd-bdsbj"]
Jul 23 00:08:55.909553 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909488    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-secret-metrics-server-client-certs\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.909553 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909521    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-client-ca-bundle\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.909696 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909555    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dbbcbdac-a55e-481f-99d6-b40966163542-metrics-server-audit-profiles\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.909696 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909618    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fnbk\" (UniqueName: \"kubernetes.io/projected/dbbcbdac-a55e-481f-99d6-b40966163542-kube-api-access-8fnbk\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.909696 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909679    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbbcbdac-a55e-481f-99d6-b40966163542-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.909804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909697    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dbbcbdac-a55e-481f-99d6-b40966163542-audit-log\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:55.909804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:55.909774    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-secret-metrics-server-tls\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011049    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbbcbdac-a55e-481f-99d6-b40966163542-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011090    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dbbcbdac-a55e-481f-99d6-b40966163542-audit-log\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-secret-metrics-server-tls\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011320 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-secret-metrics-server-client-certs\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011320 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011277    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-client-ca-bundle\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011320 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011308    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dbbcbdac-a55e-481f-99d6-b40966163542-metrics-server-audit-profiles\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fnbk\" (UniqueName: \"kubernetes.io/projected/dbbcbdac-a55e-481f-99d6-b40966163542-kube-api-access-8fnbk\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011551 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011528    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/dbbcbdac-a55e-481f-99d6-b40966163542-audit-log\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.011992 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.011949    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dbbcbdac-a55e-481f-99d6-b40966163542-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.012631 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.012606    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/dbbcbdac-a55e-481f-99d6-b40966163542-metrics-server-audit-profiles\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.014570 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.014523    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-secret-metrics-server-tls\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.014570 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.014554    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-client-ca-bundle\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.015616 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.015594    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/dbbcbdac-a55e-481f-99d6-b40966163542-secret-metrics-server-client-certs\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.018911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.018892    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fnbk\" (UniqueName: \"kubernetes.io/projected/dbbcbdac-a55e-481f-99d6-b40966163542-kube-api-access-8fnbk\") pod \"metrics-server-b76f97fd-bdsbj\" (UID: \"dbbcbdac-a55e-481f-99d6-b40966163542\") " pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.162424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.162337    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:08:56.322018 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.321993    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-b76f97fd-bdsbj"]
Jul 23 00:08:56.325695 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:56.325662    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbbcbdac_a55e_481f_99d6_b40966163542.slice/crio-f88bc0affcb65f88a34a1488cb371e50e02c1cae8ad947423b55e34034c3c169 WatchSource:0}: Error finding container f88bc0affcb65f88a34a1488cb371e50e02c1cae8ad947423b55e34034c3c169: Status 404 returned error can't find the container with id f88bc0affcb65f88a34a1488cb371e50e02c1cae8ad947423b55e34034c3c169
Jul 23 00:08:56.627041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.626991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj" event={"ID":"dbbcbdac-a55e-481f-99d6-b40966163542","Type":"ContainerStarted","Data":"f88bc0affcb65f88a34a1488cb371e50e02c1cae8ad947423b55e34034c3c169"}
Jul 23 00:08:56.629573 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.629538    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2w2sb" event={"ID":"265e6475-c359-4a73-b86c-905041a290bf","Type":"ContainerStarted","Data":"328abc34d8dca5f925f031131a54287639f380167c9db5318b397c1328a159b8"}
Jul 23 00:08:56.629699 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.629591    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2w2sb" event={"ID":"265e6475-c359-4a73-b86c-905041a290bf","Type":"ContainerStarted","Data":"cdba8cf2ad86f6b6c6388fdcd0aaee367a95a52a9df2a4c19f9f7274a75da13a"}
Jul 23 00:08:56.647103 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:56.647059    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2w2sb" podStartSLOduration=4.47085676 podStartE2EDuration="5.647044515s" totalImagesPullingTime="1.109100391s" totalInitContainerRuntime="67.087364ms" isStatefulPod=true podCreationTimestamp="2026-07-23 00:08:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:56.645131637 +0000 UTC m=+286.369446823" watchObservedRunningTime="2026-07-23 00:08:56.647044515 +0000 UTC m=+286.371359698"
Jul 23 00:08:57.877757 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.877724    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-66b57f6ff6-tlkqq"]
Jul 23 00:08:57.881325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.881303    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:57.883526 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.883508    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 00:08:57.884238 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884215    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 00:08:57.884342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884260    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 00:08:57.884342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884279    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 00:08:57.884342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dgbq4\""
Jul 23 00:08:57.884342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884315    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 00:08:57.884342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884261    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 00:08:57.884592 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.884578    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 00:08:57.889002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.888735    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b57f6ff6-tlkqq"]
Jul 23 00:08:57.931161 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.931134    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7phzq\" (UniqueName: \"kubernetes.io/projected/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-kube-api-access-7phzq\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:57.931284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.931170    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-oauth-config\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:57.931284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.931186    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-service-ca\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:57.931284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.931218    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-serving-cert\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:57.931284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.931245    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-config\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:57.931457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:57.931355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-oauth-serving-cert\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.032430 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.032400    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7phzq\" (UniqueName: \"kubernetes.io/projected/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-kube-api-access-7phzq\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.032577 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.032464    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-oauth-config\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.032577 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.032491    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-service-ca\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.032577 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.032524    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-serving-cert\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.032577 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.032546    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-config\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.032769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.032576    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-oauth-serving-cert\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.033381 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.033329    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-service-ca\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.033381 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.033371    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-config\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.033550 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.033393    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-oauth-serving-cert\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.034997 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.034972    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-oauth-config\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.035199 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.035176    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-serving-cert\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.040677 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.040647    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7phzq\" (UniqueName: \"kubernetes.io/projected/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-kube-api-access-7phzq\") pod \"console-66b57f6ff6-tlkqq\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") " pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.191588 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.191512    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:08:58.815522 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:58.815446    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-66b57f6ff6-tlkqq"]
Jul 23 00:08:58.820607 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:08:58.820579    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d5371d0_2f23_40f3_a1e8_4c33bbdcbd0c.slice/crio-c96293f27ff5e811f09e863e78f48769f031029b30f1172869e1367967a25e5b WatchSource:0}: Error finding container c96293f27ff5e811f09e863e78f48769f031029b30f1172869e1367967a25e5b: Status 404 returned error can't find the container with id c96293f27ff5e811f09e863e78f48769f031029b30f1172869e1367967a25e5b
Jul 23 00:08:59.649974 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.649932    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"}
Jul 23 00:08:59.650433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.650073    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"}
Jul 23 00:08:59.650433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.650122    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"}
Jul 23 00:08:59.650433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.650161    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"}
Jul 23 00:08:59.650433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.650200    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"}
Jul 23 00:08:59.653950 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.653634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ttkgf" event={"ID":"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23","Type":"ContainerStarted","Data":"2df730c795ff57daeb80b57a29ec261755f7444606c4e30bca21be56dcb1d729"}
Jul 23 00:08:59.653950 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.653688    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ttkgf" event={"ID":"23918e6b-3a8a-4f61-98c5-dc0f21cb1b23","Type":"ContainerStarted","Data":"ff1ebbad84ac4fbf51a46a8bc0abc0c200a5df0795561f911ee93efcb1a50d7f"}
Jul 23 00:08:59.653950 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.653906    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:08:59.658266 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.656916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj" event={"ID":"dbbcbdac-a55e-481f-99d6-b40966163542","Type":"ContainerStarted","Data":"7bdb8dbe2235b5374f22a323dc254a308ed544b6c69badc0f19c37c8de957b99"}
Jul 23 00:08:59.663117 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.662951    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"01128ff7d748a453a951043417c3d7632e33611d35fd53892b805adbb81b4df9"}
Jul 23 00:08:59.663117 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.663014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"dc1db0d51eb7df5cb115fb1ad68c9eb364276f525e00f5196b5fc96cb8153480"}
Jul 23 00:08:59.663117 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.663053    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"d63865cafd960f4c39d4f9e40a53644c7a858122678a6fcb5104cbadd53b6416"}
Jul 23 00:08:59.665314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.665292    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b57f6ff6-tlkqq" event={"ID":"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c","Type":"ContainerStarted","Data":"c96293f27ff5e811f09e863e78f48769f031029b30f1172869e1367967a25e5b"}
Jul 23 00:08:59.673249 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.671381    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ttkgf" podStartSLOduration=251.352963709 podStartE2EDuration="4m14.671364532s" totalImagesPullingTime="3.318400823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:04:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:59.669778002 +0000 UTC m=+289.394093212" watchObservedRunningTime="2026-07-23 00:08:59.671364532 +0000 UTC m=+289.395679722"
Jul 23 00:08:59.686630 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:08:59.686448    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj" podStartSLOduration=2.350774374 podStartE2EDuration="4.686431051s" totalImagesPullingTime="2.335656677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:08:59.685071866 +0000 UTC m=+289.409387053" watchObservedRunningTime="2026-07-23 00:08:59.686431051 +0000 UTC m=+289.410746225"
Jul 23 00:09:00.673060 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.673025    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"72930b12b510eb616b21c42dc46faa7b6ccaabbed2bc06a9970bc02a36bb1766"}
Jul 23 00:09:00.673515 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.673109    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"850d1b5a7893e82153433e744c75b12ca8f0ee93ebe6a646fb7bf6589edaea25"}
Jul 23 00:09:00.673515 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.673154    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" event={"ID":"69584887-f0ad-4f3f-8075-fb4c9a8e6856","Type":"ContainerStarted","Data":"05a1a59f9272f7af13ff8a2635e5701637ac6515656a05e620ead4a760cb0c9b"}
Jul 23 00:09:00.673515 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.673199    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:09:00.676974 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.676887    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerStarted","Data":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"}
Jul 23 00:09:00.695036 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.694937    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb" podStartSLOduration=1.868377389 podStartE2EDuration="6.694919734s" totalImagesPullingTime="4.826542345s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:09:00.693807174 +0000 UTC m=+290.418122379" watchObservedRunningTime="2026-07-23 00:09:00.694919734 +0000 UTC m=+290.419234918"
Jul 23 00:09:00.723608 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:00.723554    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.4637865789999998 podStartE2EDuration="8.72353555s" totalImagesPullingTime="5.240529514s" totalInitContainerRuntime="19.219457ms" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:09:00.721487452 +0000 UTC m=+290.445802636" watchObservedRunningTime="2026-07-23 00:09:00.72353555 +0000 UTC m=+290.447850735"
Jul 23 00:09:02.686704 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:02.686661    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b57f6ff6-tlkqq" event={"ID":"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c","Type":"ContainerStarted","Data":"61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4"}
Jul 23 00:09:02.703091 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:02.703036    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-66b57f6ff6-tlkqq" podStartSLOduration=2.557881706 podStartE2EDuration="5.703017591s" totalImagesPullingTime="3.145135885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:08:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:09:02.702532489 +0000 UTC m=+292.426847678" watchObservedRunningTime="2026-07-23 00:09:02.703017591 +0000 UTC m=+292.427332774"
Jul 23 00:09:06.688647 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:06.688615    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7885b4b854-lrjzb"
Jul 23 00:09:08.192008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:08.191955    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:09:08.192490 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:08.192048    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:09:08.193679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:08.193652    2558 patch_prober.go:28] interesting pod/console-66b57f6ff6-tlkqq container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused" start-of-body=
Jul 23 00:09:08.193796 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:08.193712    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-66b57f6ff6-tlkqq" podUID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" containerName="console" probeResult="failure" output="Get \"https://10.132.0.22:8443/health\": dial tcp 10.132.0.22:8443: connect: connection refused"
Jul 23 00:09:09.680154 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:09.680120    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ttkgf"
Jul 23 00:09:10.632492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:10.632461    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:09:10.632674 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:10.632536    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:09:10.638120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:10.638099    2558 kubelet.go:1648] "Image garbage collection succeeded"
Jul 23 00:09:15.199009 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:15.198965    2558 patch_prober.go:28] interesting pod/image-registry-7cfbc886f-76f7c container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 23 00:09:15.199411 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:15.199027    2558 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 23 00:09:16.162583 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:16.162551    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:09:16.162769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:16.162614    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:09:16.634472 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:16.634438    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:09:16.853017 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:16.852986    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b57f6ff6-tlkqq"]
Jul 23 00:09:23.576991 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:23.576953    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cfbc886f-76f7c"]
Jul 23 00:09:36.168411 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:36.168374    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:09:36.172358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:36.172333    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-b76f97fd-bdsbj"
Jul 23 00:09:37.827483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:37.827449    2558 generic.go:356] "Generic (PLEG): container finished" podID="29935393-8973-47ec-9c30-b7a21f0e270f" containerID="277a41f1241963eb33dab166079cad79e31dbd933973405da9ee1dbba4782b1f" exitCode=0
Jul 23 00:09:37.827782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:37.827519    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-p5kxt" event={"ID":"29935393-8973-47ec-9c30-b7a21f0e270f","Type":"ContainerDied","Data":"277a41f1241963eb33dab166079cad79e31dbd933973405da9ee1dbba4782b1f"}
Jul 23 00:09:37.827896 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:37.827881    2558 scope.go:117] "RemoveContainer" containerID="277a41f1241963eb33dab166079cad79e31dbd933973405da9ee1dbba4782b1f"
Jul 23 00:09:38.832662 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:38.832627    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-p5kxt" event={"ID":"29935393-8973-47ec-9c30-b7a21f0e270f","Type":"ContainerStarted","Data":"a75eae300ec912d8133dd652f90547fda5932fb710576bfa229ce8e5d0bfeb6c"}
Jul 23 00:09:41.875573 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:41.875505    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-66b57f6ff6-tlkqq" podUID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" containerName="console" containerID="cri-o://61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4" gracePeriod=15
Jul 23 00:09:42.119316 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.119294    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:09:42.179147 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179081    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-config\") pod \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") "
Jul 23 00:09:42.179147 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179122    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-oauth-serving-cert\") pod \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") "
Jul 23 00:09:42.179311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179167    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-oauth-config\") pod \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") "
Jul 23 00:09:42.179311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179208    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-serving-cert\") pod \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") "
Jul 23 00:09:42.179311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179284    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7phzq\" (UniqueName: \"kubernetes.io/projected/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-kube-api-access-7phzq\") pod \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") "
Jul 23 00:09:42.179458 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179313    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-service-ca\") pod \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\" (UID: \"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c\") "
Jul 23 00:09:42.179522 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179492    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-config" (OuterVolumeSpecName: "console-config") pod "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" (UID: "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:09:42.179576 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179548    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" (UID: "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:09:42.179798 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.179767    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-service-ca" (OuterVolumeSpecName: "service-ca") pod "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" (UID: "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:09:42.181387 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.181358    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-kube-api-access-7phzq" (OuterVolumeSpecName: "kube-api-access-7phzq") pod "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" (UID: "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c"). InnerVolumeSpecName "kube-api-access-7phzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:09:42.181475 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.181383    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" (UID: "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:09:42.181475 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.181402    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" (UID: "9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:09:42.279921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.279898    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7phzq\" (UniqueName: \"kubernetes.io/projected/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-kube-api-access-7phzq\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:42.279921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.279920    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:42.280050 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.279929    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:42.280050 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.279937    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-oauth-serving-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:42.280050 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.279946    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-oauth-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:42.280050 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.279954    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c-console-serving-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:42.849872 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.849812    2558 generic.go:356] "Generic (PLEG): container finished" podID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" containerID="61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4" exitCode=0
Jul 23 00:09:42.850017 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.849873    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-66b57f6ff6-tlkqq"
Jul 23 00:09:42.850017 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.849910    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b57f6ff6-tlkqq" event={"ID":"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c","Type":"ContainerDied","Data":"61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4"}
Jul 23 00:09:42.850017 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.849955    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-66b57f6ff6-tlkqq" event={"ID":"9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c","Type":"ContainerDied","Data":"c96293f27ff5e811f09e863e78f48769f031029b30f1172869e1367967a25e5b"}
Jul 23 00:09:42.850017 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.849975    2558 scope.go:117] "RemoveContainer" containerID="61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4"
Jul 23 00:09:42.858692 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.858671    2558 scope.go:117] "RemoveContainer" containerID="61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4"
Jul 23 00:09:42.858988 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:09:42.858961    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4\": container with ID starting with 61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4 not found: ID does not exist" containerID="61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4"
Jul 23 00:09:42.859084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.858993    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4"} err="failed to get container status \"61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4\": rpc error: code = NotFound desc = could not find container \"61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4\": container with ID starting with 61f1e859c910bd5c06ac780d682ebbedb0f8adc7bc48d82a59c221d46f19e0f4 not found: ID does not exist"
Jul 23 00:09:42.865745 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.865723    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-66b57f6ff6-tlkqq"]
Jul 23 00:09:42.869177 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:42.869155    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-66b57f6ff6-tlkqq"]
Jul 23 00:09:44.733618 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:44.733589    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" path="/var/lib/kubelet/pods/9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c/volumes"
Jul 23 00:09:48.597494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.597415    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa" containerName="registry" containerID="cri-o://172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4" gracePeriod=30
Jul 23 00:09:48.831314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.831290    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:09:48.877441 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.877182    2558 generic.go:356] "Generic (PLEG): container finished" podID="1b718f23-82f5-4b53-9f6b-e17f42514a82" containerID="f2456160620c9fc1a9c5bd6d67af84891805cd597cc4131d929176af1ac36737" exitCode=0
Jul 23 00:09:48.877441 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.877277    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29746080-c9kg2" event={"ID":"1b718f23-82f5-4b53-9f6b-e17f42514a82","Type":"ContainerDied","Data":"f2456160620c9fc1a9c5bd6d67af84891805cd597cc4131d929176af1ac36737"}
Jul 23 00:09:48.879293 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.879267    2558 generic.go:356] "Generic (PLEG): container finished" podID="55f33850-7ffa-4ed1-83d7-477a646618fa" containerID="172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4" exitCode=0
Jul 23 00:09:48.879421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.879303    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c"
Jul 23 00:09:48.879421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.879330    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" event={"ID":"55f33850-7ffa-4ed1-83d7-477a646618fa","Type":"ContainerDied","Data":"172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4"}
Jul 23 00:09:48.879421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.879376    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7cfbc886f-76f7c" event={"ID":"55f33850-7ffa-4ed1-83d7-477a646618fa","Type":"ContainerDied","Data":"c83b1c941cc1c60608cbd3446f1ae9fc3f7baafa038641247f88777ab5454df6"}
Jul 23 00:09:48.879421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.879408    2558 scope.go:117] "RemoveContainer" containerID="172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4"
Jul 23 00:09:48.887965 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.887949    2558 scope.go:117] "RemoveContainer" containerID="172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4"
Jul 23 00:09:48.888224 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:09:48.888208    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4\": container with ID starting with 172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4 not found: ID does not exist" containerID="172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4"
Jul 23 00:09:48.888292 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.888235    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4"} err="failed to get container status \"172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4\": rpc error: code = NotFound desc = could not find container \"172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4\": container with ID starting with 172fd564592cd5f9e1e81e4503fb9e06c13300c913ef6b152599eed3799c13e4 not found: ID does not exist"
Jul 23 00:09:48.929793 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.929775    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xszkn\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-kube-api-access-xszkn\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.929911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.929807    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-certificates\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.929911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.929856    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-bound-sa-token\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.929992 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.929926    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-trusted-ca\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.930031 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930001    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-image-registry-private-configuration\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.930077 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930060    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-installation-pull-secrets\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.930124 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930101    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/55f33850-7ffa-4ed1-83d7-477a646618fa-ca-trust-extracted\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.930176 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930136    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") pod \"55f33850-7ffa-4ed1-83d7-477a646618fa\" (UID: \"55f33850-7ffa-4ed1-83d7-477a646618fa\") "
Jul 23 00:09:48.930248 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930223    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:09:48.930429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930340    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:09:48.930429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.930386    2558 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-certificates\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:48.932456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.932430    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-kube-api-access-xszkn" (OuterVolumeSpecName: "kube-api-access-xszkn") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "kube-api-access-xszkn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:09:48.932555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.932449    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:09:48.932555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.932470    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:09:48.932555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.932525    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:09:48.932666 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.932617    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:09:48.938669 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:48.938648    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55f33850-7ffa-4ed1-83d7-477a646618fa-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "55f33850-7ffa-4ed1-83d7-477a646618fa" (UID: "55f33850-7ffa-4ed1-83d7-477a646618fa"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:09:49.030970 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.030940    2558 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-image-registry-private-configuration\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.030970 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.030970    2558 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/55f33850-7ffa-4ed1-83d7-477a646618fa-installation-pull-secrets\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.031082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.030981    2558 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/55f33850-7ffa-4ed1-83d7-477a646618fa-ca-trust-extracted\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.031082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.030990    2558 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-registry-tls\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.031082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.030998    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xszkn\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-kube-api-access-xszkn\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.031082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.031007    2558 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/55f33850-7ffa-4ed1-83d7-477a646618fa-bound-sa-token\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.031082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.031014    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/55f33850-7ffa-4ed1-83d7-477a646618fa-trusted-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:49.199297 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.199270    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7cfbc886f-76f7c"]
Jul 23 00:09:49.202845 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:49.202815    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7cfbc886f-76f7c"]
Jul 23 00:09:50.013551 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.013528    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:09:50.039196 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.039172    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b718f23-82f5-4b53-9f6b-e17f42514a82-serviceca\") pod \"1b718f23-82f5-4b53-9f6b-e17f42514a82\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") "
Jul 23 00:09:50.039353 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.039207    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm72c\" (UniqueName: \"kubernetes.io/projected/1b718f23-82f5-4b53-9f6b-e17f42514a82-kube-api-access-dm72c\") pod \"1b718f23-82f5-4b53-9f6b-e17f42514a82\" (UID: \"1b718f23-82f5-4b53-9f6b-e17f42514a82\") "
Jul 23 00:09:50.039496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.039475    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b718f23-82f5-4b53-9f6b-e17f42514a82-serviceca" (OuterVolumeSpecName: "serviceca") pod "1b718f23-82f5-4b53-9f6b-e17f42514a82" (UID: "1b718f23-82f5-4b53-9f6b-e17f42514a82"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:09:50.041369 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.041344    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b718f23-82f5-4b53-9f6b-e17f42514a82-kube-api-access-dm72c" (OuterVolumeSpecName: "kube-api-access-dm72c") pod "1b718f23-82f5-4b53-9f6b-e17f42514a82" (UID: "1b718f23-82f5-4b53-9f6b-e17f42514a82"). InnerVolumeSpecName "kube-api-access-dm72c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:09:50.139995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.139944    2558 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/1b718f23-82f5-4b53-9f6b-e17f42514a82-serviceca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:50.139995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.139964    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm72c\" (UniqueName: \"kubernetes.io/projected/1b718f23-82f5-4b53-9f6b-e17f42514a82-kube-api-access-dm72c\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:09:50.733764 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.733731    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa" path="/var/lib/kubelet/pods/55f33850-7ffa-4ed1-83d7-477a646618fa/volumes"
Jul 23 00:09:50.890192 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.890165    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29746080-c9kg2"
Jul 23 00:09:50.890334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.890195    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29746080-c9kg2" event={"ID":"1b718f23-82f5-4b53-9f6b-e17f42514a82","Type":"ContainerDied","Data":"5928788d293348118c9c4f67f676df211b6d1113250ca441f994a87efe926ae6"}
Jul 23 00:09:50.890334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:09:50.890237    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5928788d293348118c9c4f67f676df211b6d1113250ca441f994a87efe926ae6"
Jul 23 00:10:11.715423 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715389    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:10:11.715772 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715653    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="alertmanager" containerID="cri-o://96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a" gracePeriod=120
Jul 23 00:10:11.715772 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715720    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-metric" containerID="cri-o://51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693" gracePeriod=120
Jul 23 00:10:11.715926 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715765    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="prom-label-proxy" containerID="cri-o://46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051" gracePeriod=120
Jul 23 00:10:11.715926 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715789    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-web" containerID="cri-o://99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891" gracePeriod=120
Jul 23 00:10:11.715926 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715806    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy" containerID="cri-o://203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577" gracePeriod=120
Jul 23 00:10:11.715926 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.715802    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="config-reloader" containerID="cri-o://0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa" gracePeriod=120
Jul 23 00:10:11.972473 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.972424    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:11.978277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978257    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051" exitCode=0
Jul 23 00:10:11.978277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978274    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693" exitCode=0
Jul 23 00:10:11.978277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978280    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577" exitCode=0
Jul 23 00:10:11.978471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978286    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891" exitCode=0
Jul 23 00:10:11.978471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978291    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa" exitCode=0
Jul 23 00:10:11.978471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978298    2558 generic.go:356] "Generic (PLEG): container finished" podID="74822e68-3ed8-4663-837a-aa00632b0d61" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a" exitCode=0
Jul 23 00:10:11.978471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978323    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:11.978471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978324    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"}
Jul 23 00:10:11.978471 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978465    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"}
Jul 23 00:10:11.978651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"}
Jul 23 00:10:11.978651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978528    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"}
Jul 23 00:10:11.978651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978534    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:11.978651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978566    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"}
Jul 23 00:10:11.978651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978603    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"}
Jul 23 00:10:11.978651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.978641    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"74822e68-3ed8-4663-837a-aa00632b0d61","Type":"ContainerDied","Data":"c00a5b16fabc9afe7384ac0c93df17e0ad16c88806bf4a6738365d91c00a3886"}
Jul 23 00:10:11.987505 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.987488    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:11.994321 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:11.994304    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.001386 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.001357    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.008281 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.008263    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.016912 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.016894    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.024030 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.024011    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.030508 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.030493    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.030759 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.030742    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.030850 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.030771    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"} err="failed to get container status \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist"
Jul 23 00:10:12.030850 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.030793    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.031202 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.031185    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.031261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.031215    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"} err="failed to get container status \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist"
Jul 23 00:10:12.031261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.031230    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.031473 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.031450    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.031508 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.031484    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"} err="failed to get container status \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist"
Jul 23 00:10:12.031544 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.031506    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.031714 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.031695    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.031749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.031722    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"} err="failed to get container status \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist"
Jul 23 00:10:12.031749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.031740    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.032017 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.032000    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.032088 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032024    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"} err="failed to get container status \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist"
Jul 23 00:10:12.032088 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032043    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.032269 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.032246    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.032312 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032278    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"} err="failed to get container status \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist"
Jul 23 00:10:12.032312 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032297    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.032502 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:10:12.032485    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.032540 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032507    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"} err="failed to get container status \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist"
Jul 23 00:10:12.032540 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032524    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.032733 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032710    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"} err="failed to get container status \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist"
Jul 23 00:10:12.032733 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032733    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.032937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032920    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"} err="failed to get container status \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist"
Jul 23 00:10:12.032937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.032936    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.033186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033166    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"} err="failed to get container status \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist"
Jul 23 00:10:12.033246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033188    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.033414 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033394    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"} err="failed to get container status \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist"
Jul 23 00:10:12.033463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033415    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.033647 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033629    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"} err="failed to get container status \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist"
Jul 23 00:10:12.033686 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033649    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.033905 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033882    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"} err="failed to get container status \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist"
Jul 23 00:10:12.033973 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.033907    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.034194 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034177    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"} err="failed to get container status \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist"
Jul 23 00:10:12.034247 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034195    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.034431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034411    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"} err="failed to get container status \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist"
Jul 23 00:10:12.034499 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034433    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.034655 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034632    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"} err="failed to get container status \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist"
Jul 23 00:10:12.034655 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034654    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.034861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034821    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"} err="failed to get container status \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist"
Jul 23 00:10:12.034861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.034859    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.035071 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035052    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"} err="failed to get container status \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist"
Jul 23 00:10:12.035128 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035071    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.035249 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035234    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"} err="failed to get container status \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist"
Jul 23 00:10:12.035314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035249    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.035453 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035436    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"} err="failed to get container status \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist"
Jul 23 00:10:12.035512 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035454    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.035631 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035613    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"} err="failed to get container status \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist"
Jul 23 00:10:12.035685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035632    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.035861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035830    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"} err="failed to get container status \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist"
Jul 23 00:10:12.035932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.035863    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.036048 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036028    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"} err="failed to get container status \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist"
Jul 23 00:10:12.036107 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036048    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.036225 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036210    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"} err="failed to get container status \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist"
Jul 23 00:10:12.036285 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036224    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.036434 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036417    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"} err="failed to get container status \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist"
Jul 23 00:10:12.036496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036434    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.036621 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036606    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"} err="failed to get container status \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist"
Jul 23 00:10:12.036681 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036621    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.036809 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036794    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"} err="failed to get container status \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist"
Jul 23 00:10:12.036992 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.036808    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.037082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037064    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"} err="failed to get container status \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist"
Jul 23 00:10:12.037145 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037082    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.037254 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037238    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"} err="failed to get container status \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist"
Jul 23 00:10:12.037313 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037254    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.037463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037445    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"} err="failed to get container status \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist"
Jul 23 00:10:12.037522 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037464    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.037654 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037639    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"} err="failed to get container status \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist"
Jul 23 00:10:12.037715 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037653    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.037822 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037804    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"} err="failed to get container status \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist"
Jul 23 00:10:12.037893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037822    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.037991 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037973    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"} err="failed to get container status \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist"
Jul 23 00:10:12.038046 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.037991    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.038236 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038218    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"} err="failed to get container status \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist"
Jul 23 00:10:12.038295 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038236    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.038475 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038459    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"} err="failed to get container status \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist"
Jul 23 00:10:12.038537 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038475    2558 scope.go:117] "RemoveContainer" containerID="46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"
Jul 23 00:10:12.038701 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038684    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051"} err="failed to get container status \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": rpc error: code = NotFound desc = could not find container \"46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051\": container with ID starting with 46c420db9f4849bc4094a21dedb5e22f87c68858b153422fb313bab6d2d83051 not found: ID does not exist"
Jul 23 00:10:12.038759 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038702    2558 scope.go:117] "RemoveContainer" containerID="51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"
Jul 23 00:10:12.038950 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038931    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693"} err="failed to get container status \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": rpc error: code = NotFound desc = could not find container \"51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693\": container with ID starting with 51f64ee2e3964c910e9aa4efe575511f9408a5fbe5267f8696b313c5948d1693 not found: ID does not exist"
Jul 23 00:10:12.039025 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.038950    2558 scope.go:117] "RemoveContainer" containerID="203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"
Jul 23 00:10:12.039160 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039142    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577"} err="failed to get container status \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": rpc error: code = NotFound desc = could not find container \"203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577\": container with ID starting with 203705d748f0d65b1f1cbdb3767baccc62b1b6142b6e1b7ed50a93e6ed795577 not found: ID does not exist"
Jul 23 00:10:12.039224 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039161    2558 scope.go:117] "RemoveContainer" containerID="99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"
Jul 23 00:10:12.039343 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039328    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891"} err="failed to get container status \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": rpc error: code = NotFound desc = could not find container \"99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891\": container with ID starting with 99efda9137237d71e611b64e1689ab467933aad4524e10b81d4a86a3c4dbb891 not found: ID does not exist"
Jul 23 00:10:12.039401 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039342    2558 scope.go:117] "RemoveContainer" containerID="0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"
Jul 23 00:10:12.039512 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039497    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa"} err="failed to get container status \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": rpc error: code = NotFound desc = could not find container \"0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa\": container with ID starting with 0a3c3e82e7d12f74d93fc1e2391c3cdc3c5ad8b641730ef6becff45720f963aa not found: ID does not exist"
Jul 23 00:10:12.039567 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039512    2558 scope.go:117] "RemoveContainer" containerID="96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"
Jul 23 00:10:12.039712 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039697    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a"} err="failed to get container status \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": rpc error: code = NotFound desc = could not find container \"96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a\": container with ID starting with 96d25d81a48e66841eb21bb460f67379f2382ae4e7327adb41e3705cb5dcb93a not found: ID does not exist"
Jul 23 00:10:12.039774 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039711    2558 scope.go:117] "RemoveContainer" containerID="91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"
Jul 23 00:10:12.039921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.039906    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720"} err="failed to get container status \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": rpc error: code = NotFound desc = could not find container \"91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720\": container with ID starting with 91a20e138dad3578475b123d0711ca77c1c71e182cfd7ae1772f72b1937d1720 not found: ID does not exist"
Jul 23 00:10:12.093468 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093450    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093478    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-metrics-client-ca\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093496    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-web-config\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093512    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-tls-assets\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093534    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rfdj\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-kube-api-access-8rfdj\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093552    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-config-volume\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093568    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-config-out\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093626    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093659    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-metric\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093694    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093732    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-main-db\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.093799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093769    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-cluster-tls-config\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.094132 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093805    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-web\") pod \"74822e68-3ed8-4663-837a-aa00632b0d61\" (UID: \"74822e68-3ed8-4663-837a-aa00632b0d61\") "
Jul 23 00:10:12.094132 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.093895    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:10:12.094132 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.094110    2558 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-metrics-client-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.094299 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.094269    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:10:12.094945 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.094920    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:10:12.096361 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.096340    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-config-out" (OuterVolumeSpecName: "config-out") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:10:12.096821 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.096789    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.096821 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.096792    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-kube-api-access-8rfdj" (OuterVolumeSpecName: "kube-api-access-8rfdj") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "kube-api-access-8rfdj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:10:12.096966 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.096821    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.097166 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.097143    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.097238 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.097224    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:10:12.097558 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.097541    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.098254 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.098240    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-config-volume" (OuterVolumeSpecName: "config-volume") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.101651 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.101619    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.107145 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.107125    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-web-config" (OuterVolumeSpecName: "web-config") pod "74822e68-3ed8-4663-837a-aa00632b0d61" (UID: "74822e68-3ed8-4663-837a-aa00632b0d61"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:10:12.194590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194567    2558 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-cluster-tls-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194591    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194605    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-main-tls\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194618    2558 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-web-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194629    2558 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-tls-assets\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194641    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rfdj\" (UniqueName: \"kubernetes.io/projected/74822e68-3ed8-4663-837a-aa00632b0d61-kube-api-access-8rfdj\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194653    2558 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-config-volume\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194664    2558 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-config-out\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194933 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194678    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194933 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194693    2558 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/74822e68-3ed8-4663-837a-aa00632b0d61-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194933 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194708    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.194933 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.194722    2558 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/74822e68-3ed8-4663-837a-aa00632b0d61-alertmanager-main-db\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:10:12.301678 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.301655    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:10:12.306946 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.306925    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:10:12.330584 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330562    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:10:12.330827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330815    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa" containerName="registry"
Jul 23 00:10:12.330885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330830    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa" containerName="registry"
Jul 23 00:10:12.330885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330857    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy"
Jul 23 00:10:12.330885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330863    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy"
Jul 23 00:10:12.330885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330869    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="prom-label-proxy"
Jul 23 00:10:12.330885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330875    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="prom-label-proxy"
Jul 23 00:10:12.330885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330884    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="config-reloader"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330890    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="config-reloader"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330897    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" containerName="console"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330902    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" containerName="console"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330908    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="init-config-reloader"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330914    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="init-config-reloader"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330919    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="alertmanager"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330924    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="alertmanager"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330931    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-metric"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330936    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-metric"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330945    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b718f23-82f5-4b53-9f6b-e17f42514a82" containerName="image-pruner"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330950    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b718f23-82f5-4b53-9f6b-e17f42514a82" containerName="image-pruner"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330957    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-web"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.330962    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-web"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331007    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331014    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="config-reloader"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331019    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="alertmanager"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331025    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b718f23-82f5-4b53-9f6b-e17f42514a82" containerName="image-pruner"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331030    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="55f33850-7ffa-4ed1-83d7-477a646618fa" containerName="registry"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331036    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-web"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331041    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="prom-label-proxy"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331047    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" containerName="kube-rbac-proxy-metric"
Jul 23 00:10:12.331059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.331052    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d5371d0-2f23-40f3-a1e8-4c33bbdcbd0c" containerName="console"
Jul 23 00:10:12.335858 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.335827    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.338024 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338002    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 23 00:10:12.338131 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338069    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 23 00:10:12.338131 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338106    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-tkvgj\""
Jul 23 00:10:12.338244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338205    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 23 00:10:12.338368 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338352    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 23 00:10:12.338431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338379    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 23 00:10:12.338431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338417    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 23 00:10:12.338648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338634    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 23 00:10:12.338717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.338634    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 23 00:10:12.343754 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.343735    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 23 00:10:12.352758 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.352731    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:10:12.396130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396107    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396144    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-config-volume\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396160    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396176    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396223    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396270    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7wrr\" (UniqueName: \"kubernetes.io/projected/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-kube-api-access-w7wrr\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396297    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-web-config\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396313    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396332    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-config-out\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396353    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396375    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396394    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.396549 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.396418    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497458 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497420    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497567 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497476    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-config-volume\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497618 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497596    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497653 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497718 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497691    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7wrr\" (UniqueName: \"kubernetes.io/projected/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-kube-api-access-w7wrr\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497756 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497723    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-web-config\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497756 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497750    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497878 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497774    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-config-out\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497878 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497796    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.497878 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497831    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.498014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497893    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.498014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.497920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.498603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.498575    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.498603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.498595    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.499477 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.499451    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.500585 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.500550    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-web-config\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.500671 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.500586    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.501262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.500869    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-config-volume\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.501262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.500883    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.501262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.500970    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.501262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.501211    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.501262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.501217    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.501262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.501243    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.502314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.502295    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-config-out\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.505230 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.505206    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7wrr\" (UniqueName: \"kubernetes.io/projected/e35a6dc5-45d5-424d-8a0c-8d9712cbb10a-kube-api-access-w7wrr\") pod \"alertmanager-main-0\" (UID: \"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.645485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.645468    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 23 00:10:12.736146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.736113    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74822e68-3ed8-4663-837a-aa00632b0d61" path="/var/lib/kubelet/pods/74822e68-3ed8-4663-837a-aa00632b0d61/volumes"
Jul 23 00:10:12.781855 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.781796    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 23 00:10:12.784078 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:10:12.784049    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode35a6dc5_45d5_424d_8a0c_8d9712cbb10a.slice/crio-15530aad23f3a8a231d078fb4317909451f4859886fc4a4a2f1e86b5e2f40cf4 WatchSource:0}: Error finding container 15530aad23f3a8a231d078fb4317909451f4859886fc4a4a2f1e86b5e2f40cf4: Status 404 returned error can't find the container with id 15530aad23f3a8a231d078fb4317909451f4859886fc4a4a2f1e86b5e2f40cf4
Jul 23 00:10:12.984556 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.984496    2558 generic.go:356] "Generic (PLEG): container finished" podID="e35a6dc5-45d5-424d-8a0c-8d9712cbb10a" containerID="8fc65d74a2d97c7055942ee7800b22640e6066f2724e7ffe2e784ffdd397d67c" exitCode=0
Jul 23 00:10:12.984556 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.984534    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerDied","Data":"8fc65d74a2d97c7055942ee7800b22640e6066f2724e7ffe2e784ffdd397d67c"}
Jul 23 00:10:12.984816 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:12.984584    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"15530aad23f3a8a231d078fb4317909451f4859886fc4a4a2f1e86b5e2f40cf4"}
Jul 23 00:10:13.995749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:13.995674    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"cf70919cc1d28b56c74ceee3af34ada36bfba33d1358ac40b71dea9d0c7142c8"}
Jul 23 00:10:13.995749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:13.995736    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"6b87c1d4a9310ae0c5b0549abb1b32c91c3d8e0497496b352d759d42db6d69bc"}
Jul 23 00:10:13.996152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:13.995763    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"181865e9460f64b8890116edbf24f0b3a7e6aeaa9a7cb73a66c056df6fbf4bfe"}
Jul 23 00:10:13.996152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:13.995787    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"8ae628de36f7bd2c9d7bc491277aa27f835c388179c28546f0152b18c7d21e0a"}
Jul 23 00:10:13.996152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:13.995810    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"5b7b98517e475ce429bfdc3ba19f1cb124ce184e4f323ab5df44e2fe4d6e7584"}
Jul 23 00:10:13.996152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:13.995853    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e35a6dc5-45d5-424d-8a0c-8d9712cbb10a","Type":"ContainerStarted","Data":"6f1fd1aadf9a616d9cad7ea9fde02081edbbd3175e3ad09e00dbec7371b958ac"}
Jul 23 00:10:14.020643 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:14.020583    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.004680584 podStartE2EDuration="2.02056428s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.883696ms" isStatefulPod=false podCreationTimestamp="2026-07-23 00:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:10:14.019363833 +0000 UTC m=+363.743679017" watchObservedRunningTime="2026-07-23 00:10:14.02056428 +0000 UTC m=+363.744879468"
Jul 23 00:10:15.790274 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.790238    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"]
Jul 23 00:10:15.793844 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.793815    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.796348 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.796328    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g5vbg\""
Jul 23 00:10:15.796499 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.796433    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 23 00:10:15.796499 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.796439    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 23 00:10:15.796634 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.796622    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 23 00:10:15.796729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.796688    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 23 00:10:15.796929 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.796916    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 23 00:10:15.801374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.801289    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 23 00:10:15.804358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.804339    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"]
Jul 23 00:10:15.923140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923159    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923189    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-secret-telemeter-client\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923216    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-telemeter-client-tls\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923237    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-metrics-client-ca\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923254    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvpc5\" (UniqueName: \"kubernetes.io/projected/16d07331-b5a0-442b-8281-0a987f01c56a-kube-api-access-tvpc5\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923365    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-serving-certs-ca-bundle\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:15.923429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:15.923399    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-federate-client-tls\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.023832 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.023806    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.023979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.023857    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-secret-telemeter-client\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.023979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.023876    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-telemeter-client-tls\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.023979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.023906    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-metrics-client-ca\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.023979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.023929    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvpc5\" (UniqueName: \"kubernetes.io/projected/16d07331-b5a0-442b-8281-0a987f01c56a-kube-api-access-tvpc5\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.024207 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.023999    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-serving-certs-ca-bundle\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.024207 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.024022    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-federate-client-tls\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.024207 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.024056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.024788 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.024759    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-metrics-client-ca\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.024969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.024759    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-serving-certs-ca-bundle\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.024969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.024863    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16d07331-b5a0-442b-8281-0a987f01c56a-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.026653 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.026625    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.026737 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.026698    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-secret-telemeter-client\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.026880 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.026863    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-telemeter-client-tls\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.027093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.027071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/16d07331-b5a0-442b-8281-0a987f01c56a-federate-client-tls\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.031156 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.031137    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvpc5\" (UniqueName: \"kubernetes.io/projected/16d07331-b5a0-442b-8281-0a987f01c56a-kube-api-access-tvpc5\") pod \"telemeter-client-6cfdf887f6-bpv9g\" (UID: \"16d07331-b5a0-442b-8281-0a987f01c56a\") " pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.104371 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.104344    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"
Jul 23 00:10:16.234659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.234638    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g"]
Jul 23 00:10:16.237013 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:10:16.236981    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d07331_b5a0_442b_8281_0a987f01c56a.slice/crio-7ee3f7529fd6791b53b5ef5729116938f6c026fca39e4e6007c983b93c627fdb WatchSource:0}: Error finding container 7ee3f7529fd6791b53b5ef5729116938f6c026fca39e4e6007c983b93c627fdb: Status 404 returned error can't find the container with id 7ee3f7529fd6791b53b5ef5729116938f6c026fca39e4e6007c983b93c627fdb
Jul 23 00:10:16.239151 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:16.239136    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:10:17.008700 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:17.008665    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g" event={"ID":"16d07331-b5a0-442b-8281-0a987f01c56a","Type":"ContainerStarted","Data":"7ee3f7529fd6791b53b5ef5729116938f6c026fca39e4e6007c983b93c627fdb"}
Jul 23 00:10:19.023696 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.023615    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g" event={"ID":"16d07331-b5a0-442b-8281-0a987f01c56a","Type":"ContainerStarted","Data":"2fd530fc53527aa826307318e09f7a8ea6e3f3a7ebdc65e978de2f5614c54233"}
Jul 23 00:10:19.023696 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.023673    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g" event={"ID":"16d07331-b5a0-442b-8281-0a987f01c56a","Type":"ContainerStarted","Data":"cdd3d87ff30cad3b0b1d0e1961450b00fe4038d1782f0a68b03a0a53cded72bf"}
Jul 23 00:10:19.024154 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.023699    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g" event={"ID":"16d07331-b5a0-442b-8281-0a987f01c56a","Type":"ContainerStarted","Data":"5da6b000128401b153ad29f7b2604679324395545fdacf113195a83297bb3e5c"}
Jul 23 00:10:19.075801 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.075753    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6cfdf887f6-bpv9g" podStartSLOduration=1.5675009009999998 podStartE2EDuration="4.075739345s" totalImagesPullingTime="2.508238444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:10:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:10:19.074772194 +0000 UTC m=+368.799087378" watchObservedRunningTime="2026-07-23 00:10:19.075739345 +0000 UTC m=+368.800054529"
Jul 23 00:10:19.813289 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.813254    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cb588dcdd-s52bm"]
Jul 23 00:10:19.816610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.816591    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.819660 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.819625    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dgbq4\""
Jul 23 00:10:19.819660 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.819635    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 23 00:10:19.819861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.819663    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 23 00:10:19.819861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.819711    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 23 00:10:19.819861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.819653    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 23 00:10:19.820066 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.820047    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 23 00:10:19.820127 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.820087    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 23 00:10:19.820412 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.820389    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 23 00:10:19.824445 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.824424    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 23 00:10:19.827029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.827006    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb588dcdd-s52bm"]
Jul 23 00:10:19.856153 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856128    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-oauth-serving-cert\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.856239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856167    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgnv\" (UniqueName: \"kubernetes.io/projected/62b2dca4-4a6b-42ef-9254-cb5f7f661295-kube-api-access-thgnv\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.856239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856190    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-service-ca\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.856239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856235    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-config\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.856347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856271    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-oauth-config\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.856347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856287    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-trusted-ca-bundle\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.856465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.856446    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-serving-cert\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957508 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957485    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgnv\" (UniqueName: \"kubernetes.io/projected/62b2dca4-4a6b-42ef-9254-cb5f7f661295-kube-api-access-thgnv\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957583 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957517    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-service-ca\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957583 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-config\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957666 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957579    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-oauth-config\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957666 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957606    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-trusted-ca-bundle\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957666 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957633    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-serving-cert\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.957822 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.957704    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-oauth-serving-cert\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.958359 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.958328    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-service-ca\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.958494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.958470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-oauth-serving-cert\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.958494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.958489    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-trusted-ca-bundle\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.958612 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.958573    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-config\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.960114 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.960092    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-oauth-config\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.960248 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.960229    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-serving-cert\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:19.964138 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:19.964119    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgnv\" (UniqueName: \"kubernetes.io/projected/62b2dca4-4a6b-42ef-9254-cb5f7f661295-kube-api-access-thgnv\") pod \"console-7cb588dcdd-s52bm\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") " pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:20.128577 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:20.128512    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:20.270481 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:20.270458    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cb588dcdd-s52bm"]
Jul 23 00:10:20.272423 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:10:20.272391    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b2dca4_4a6b_42ef_9254_cb5f7f661295.slice/crio-f5ac6da8d4e37200382c9c4bab76e14f64b75c2c3bef563ab2032d346ee5427f WatchSource:0}: Error finding container f5ac6da8d4e37200382c9c4bab76e14f64b75c2c3bef563ab2032d346ee5427f: Status 404 returned error can't find the container with id f5ac6da8d4e37200382c9c4bab76e14f64b75c2c3bef563ab2032d346ee5427f
Jul 23 00:10:21.034802 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:21.034765    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb588dcdd-s52bm" event={"ID":"62b2dca4-4a6b-42ef-9254-cb5f7f661295","Type":"ContainerStarted","Data":"6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846"}
Jul 23 00:10:21.034986 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:21.034821    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb588dcdd-s52bm" event={"ID":"62b2dca4-4a6b-42ef-9254-cb5f7f661295","Type":"ContainerStarted","Data":"f5ac6da8d4e37200382c9c4bab76e14f64b75c2c3bef563ab2032d346ee5427f"}
Jul 23 00:10:21.054571 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:21.054520    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7cb588dcdd-s52bm" podStartSLOduration=2.054505067 podStartE2EDuration="2.054505067s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:10:21.052729195 +0000 UTC m=+370.777044393" watchObservedRunningTime="2026-07-23 00:10:21.054505067 +0000 UTC m=+370.778820251"
Jul 23 00:10:30.128865 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:30.128819    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:30.129227 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:30.128890    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:30.133495 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:30.133474    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:10:31.080741 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:10:31.080713    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:11:39.155916 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.155830    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77c98d4b46-5llz2"]
Jul 23 00:11:39.159372 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.159351    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.170120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.170099    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c98d4b46-5llz2"]
Jul 23 00:11:39.229384 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229350    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-service-ca\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.229496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-config\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.229496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229471    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-oauth-serving-cert\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.229613 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-oauth-config\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.229613 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229563    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-serving-cert\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.229613 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229587    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqbz7\" (UniqueName: \"kubernetes.io/projected/11470f6a-9f51-4771-b3a2-8cd34b99b91f-kube-api-access-gqbz7\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.229719 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.229613    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-trusted-ca-bundle\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330536 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-oauth-config\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330547    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-serving-cert\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330563    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqbz7\" (UniqueName: \"kubernetes.io/projected/11470f6a-9f51-4771-b3a2-8cd34b99b91f-kube-api-access-gqbz7\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330581    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-trusted-ca-bundle\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330608    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-service-ca\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330640    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-config\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.330939 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.330693    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-oauth-serving-cert\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.331387 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.331364    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-config\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.331496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.331436    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-oauth-serving-cert\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.331558 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.331520    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-service-ca\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.331617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.331587    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-trusted-ca-bundle\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.333081 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.333062    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-serving-cert\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.333183 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.333134    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-oauth-config\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.337330 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.337308    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqbz7\" (UniqueName: \"kubernetes.io/projected/11470f6a-9f51-4771-b3a2-8cd34b99b91f-kube-api-access-gqbz7\") pod \"console-77c98d4b46-5llz2\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") " pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.469275 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.469219    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:39.592870 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:39.592827    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77c98d4b46-5llz2"]
Jul 23 00:11:39.595468 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:11:39.595437    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11470f6a_9f51_4771_b3a2_8cd34b99b91f.slice/crio-80061469d305a019ad02bf031aa240d5cd544fc539f4e31654e9aac73a233cd8 WatchSource:0}: Error finding container 80061469d305a019ad02bf031aa240d5cd544fc539f4e31654e9aac73a233cd8: Status 404 returned error can't find the container with id 80061469d305a019ad02bf031aa240d5cd544fc539f4e31654e9aac73a233cd8
Jul 23 00:11:40.370008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:40.369977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c98d4b46-5llz2" event={"ID":"11470f6a-9f51-4771-b3a2-8cd34b99b91f","Type":"ContainerStarted","Data":"0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22"}
Jul 23 00:11:40.370441 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:40.370024    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c98d4b46-5llz2" event={"ID":"11470f6a-9f51-4771-b3a2-8cd34b99b91f","Type":"ContainerStarted","Data":"80061469d305a019ad02bf031aa240d5cd544fc539f4e31654e9aac73a233cd8"}
Jul 23 00:11:40.400623 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:40.400567    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77c98d4b46-5llz2" podStartSLOduration=1.400553718 podStartE2EDuration="1.400553718s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:11:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:11:40.400277739 +0000 UTC m=+450.124592941" watchObservedRunningTime="2026-07-23 00:11:40.400553718 +0000 UTC m=+450.124868902"
Jul 23 00:11:49.470187 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:49.470152    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:49.470698 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:49.470216    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:49.475332 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:49.475303    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:50.415080 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:50.415050    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:11:50.459438 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:11:50.459409    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb588dcdd-s52bm"]
Jul 23 00:12:14.806504 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.806465    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"]
Jul 23 00:12:14.810424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.810404    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.812554 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.812537    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Jul 23 00:12:14.813328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.813311    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Jul 23 00:12:14.813425 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.813405    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-qt6wm\""
Jul 23 00:12:14.817218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.817197    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"]
Jul 23 00:12:14.884231 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.884207    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.884328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.884236    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gg7t\" (UniqueName: \"kubernetes.io/projected/760c0efa-fb24-454c-854e-587a9d8c0b3e-kube-api-access-4gg7t\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.884328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.884258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.984578 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.984557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.984663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.984584    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gg7t\" (UniqueName: \"kubernetes.io/projected/760c0efa-fb24-454c-854e-587a9d8c0b3e-kube-api-access-4gg7t\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.984663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.984604    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.984977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.984959    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.985047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.984989    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:14.992659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:14.992637    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gg7t\" (UniqueName: \"kubernetes.io/projected/760c0efa-fb24-454c-854e-587a9d8c0b3e-kube-api-access-4gg7t\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:15.120232 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.120211    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:15.245068 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.245047    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"]
Jul 23 00:12:15.246991 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:12:15.246964    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod760c0efa_fb24_454c_854e_587a9d8c0b3e.slice/crio-257d95f9a2b18356934d9d4967ed78f61ee4ff408037ddd0958c640e52092652 WatchSource:0}: Error finding container 257d95f9a2b18356934d9d4967ed78f61ee4ff408037ddd0958c640e52092652: Status 404 returned error can't find the container with id 257d95f9a2b18356934d9d4967ed78f61ee4ff408037ddd0958c640e52092652
Jul 23 00:12:15.482186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.482117    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7cb588dcdd-s52bm" podUID="62b2dca4-4a6b-42ef-9254-cb5f7f661295" containerName="console" containerID="cri-o://6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846" gracePeriod=15
Jul 23 00:12:15.517406 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.517379    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b" event={"ID":"760c0efa-fb24-454c-854e-587a9d8c0b3e","Type":"ContainerStarted","Data":"257d95f9a2b18356934d9d4967ed78f61ee4ff408037ddd0958c640e52092652"}
Jul 23 00:12:15.701895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.701870    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:12:15.792799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792743    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-service-ca\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.792799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792770    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thgnv\" (UniqueName: \"kubernetes.io/projected/62b2dca4-4a6b-42ef-9254-cb5f7f661295-kube-api-access-thgnv\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.792799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792801    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-oauth-serving-cert\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.793015 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792873    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-config\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.793015 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792892    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-oauth-config\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.793015 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792932    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-trusted-ca-bundle\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.793015 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.792958    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-serving-cert\") pod \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\" (UID: \"62b2dca4-4a6b-42ef-9254-cb5f7f661295\") "
Jul 23 00:12:15.793312 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.793215    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-service-ca" (OuterVolumeSpecName: "service-ca") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:12:15.793382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.793219    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:12:15.793382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.793348    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-config" (OuterVolumeSpecName: "console-config") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:12:15.793465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.793404    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:12:15.794881 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.794854    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62b2dca4-4a6b-42ef-9254-cb5f7f661295-kube-api-access-thgnv" (OuterVolumeSpecName: "kube-api-access-thgnv") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "kube-api-access-thgnv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:12:15.794965 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.794949    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:12:15.795008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.794980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "62b2dca4-4a6b-42ef-9254-cb5f7f661295" (UID: "62b2dca4-4a6b-42ef-9254-cb5f7f661295"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:12:15.894031 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894012    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-serving-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:15.894031 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894033    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:15.894375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894043    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thgnv\" (UniqueName: \"kubernetes.io/projected/62b2dca4-4a6b-42ef-9254-cb5f7f661295-kube-api-access-thgnv\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:15.894375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894054    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-oauth-serving-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:15.894375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894064    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:15.894375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894072    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/62b2dca4-4a6b-42ef-9254-cb5f7f661295-console-oauth-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:15.894375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:15.894081    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62b2dca4-4a6b-42ef-9254-cb5f7f661295-trusted-ca-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:16.522632 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.522596    2558 generic.go:356] "Generic (PLEG): container finished" podID="62b2dca4-4a6b-42ef-9254-cb5f7f661295" containerID="6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846" exitCode=0
Jul 23 00:12:16.522781 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.522638    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cb588dcdd-s52bm"
Jul 23 00:12:16.522781 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.522679    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb588dcdd-s52bm" event={"ID":"62b2dca4-4a6b-42ef-9254-cb5f7f661295","Type":"ContainerDied","Data":"6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846"}
Jul 23 00:12:16.522781 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.522729    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cb588dcdd-s52bm" event={"ID":"62b2dca4-4a6b-42ef-9254-cb5f7f661295","Type":"ContainerDied","Data":"f5ac6da8d4e37200382c9c4bab76e14f64b75c2c3bef563ab2032d346ee5427f"}
Jul 23 00:12:16.522781 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.522750    2558 scope.go:117] "RemoveContainer" containerID="6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846"
Jul 23 00:12:16.531514 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.531501    2558 scope.go:117] "RemoveContainer" containerID="6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846"
Jul 23 00:12:16.531728 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:12:16.531712    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846\": container with ID starting with 6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846 not found: ID does not exist" containerID="6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846"
Jul 23 00:12:16.531779 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.531734    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846"} err="failed to get container status \"6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846\": rpc error: code = NotFound desc = could not find container \"6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846\": container with ID starting with 6e7d9e717840a093ec05984d27276979de31072ed26279caa792a6aa9e618846 not found: ID does not exist"
Jul 23 00:12:16.542962 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.542939    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cb588dcdd-s52bm"]
Jul 23 00:12:16.548721 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.548701    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cb588dcdd-s52bm"]
Jul 23 00:12:16.734392 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:16.734367    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62b2dca4-4a6b-42ef-9254-cb5f7f661295" path="/var/lib/kubelet/pods/62b2dca4-4a6b-42ef-9254-cb5f7f661295/volumes"
Jul 23 00:12:23.555477 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:23.555449    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b" event={"ID":"760c0efa-fb24-454c-854e-587a9d8c0b3e","Type":"ContainerStarted","Data":"5c17f94331da96c460bb83c48aa64a14b0a4e3c7c637c038fc6224f9e1ba62fa"}
Jul 23 00:12:24.560914 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:24.560878    2558 generic.go:356] "Generic (PLEG): container finished" podID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerID="5c17f94331da96c460bb83c48aa64a14b0a4e3c7c637c038fc6224f9e1ba62fa" exitCode=0
Jul 23 00:12:24.561350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:24.560966    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b" event={"ID":"760c0efa-fb24-454c-854e-587a9d8c0b3e","Type":"ContainerDied","Data":"5c17f94331da96c460bb83c48aa64a14b0a4e3c7c637c038fc6224f9e1ba62fa"}
Jul 23 00:12:26.571091 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:26.571063    2558 generic.go:356] "Generic (PLEG): container finished" podID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerID="0faf42e28a5364df836085c69d694597dedb46485e8e09eee7bbbd1dce51ee41" exitCode=0
Jul 23 00:12:26.571381 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:26.571129    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b" event={"ID":"760c0efa-fb24-454c-854e-587a9d8c0b3e","Type":"ContainerDied","Data":"0faf42e28a5364df836085c69d694597dedb46485e8e09eee7bbbd1dce51ee41"}
Jul 23 00:12:35.615155 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:35.615119    2558 generic.go:356] "Generic (PLEG): container finished" podID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerID="e746942742bb0d3041f513622596e600e8c3029c21efe42bd61f1a9d137179c2" exitCode=0
Jul 23 00:12:35.615603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:35.615205    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b" event={"ID":"760c0efa-fb24-454c-854e-587a9d8c0b3e","Type":"ContainerDied","Data":"e746942742bb0d3041f513622596e600e8c3029c21efe42bd61f1a9d137179c2"}
Jul 23 00:12:36.749756 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.749734    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:36.865093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.865067    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-util\") pod \"760c0efa-fb24-454c-854e-587a9d8c0b3e\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") "
Jul 23 00:12:36.865197 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.865104    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gg7t\" (UniqueName: \"kubernetes.io/projected/760c0efa-fb24-454c-854e-587a9d8c0b3e-kube-api-access-4gg7t\") pod \"760c0efa-fb24-454c-854e-587a9d8c0b3e\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") "
Jul 23 00:12:36.865197 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.865138    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-bundle\") pod \"760c0efa-fb24-454c-854e-587a9d8c0b3e\" (UID: \"760c0efa-fb24-454c-854e-587a9d8c0b3e\") "
Jul 23 00:12:36.865755 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.865732    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-bundle" (OuterVolumeSpecName: "bundle") pod "760c0efa-fb24-454c-854e-587a9d8c0b3e" (UID: "760c0efa-fb24-454c-854e-587a9d8c0b3e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:12:36.867251 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.867222    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/760c0efa-fb24-454c-854e-587a9d8c0b3e-kube-api-access-4gg7t" (OuterVolumeSpecName: "kube-api-access-4gg7t") pod "760c0efa-fb24-454c-854e-587a9d8c0b3e" (UID: "760c0efa-fb24-454c-854e-587a9d8c0b3e"). InnerVolumeSpecName "kube-api-access-4gg7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:12:36.869596 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.869553    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-util" (OuterVolumeSpecName: "util") pod "760c0efa-fb24-454c-854e-587a9d8c0b3e" (UID: "760c0efa-fb24-454c-854e-587a9d8c0b3e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:12:36.966198 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.966176    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-util\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:36.966198 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.966197    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gg7t\" (UniqueName: \"kubernetes.io/projected/760c0efa-fb24-454c-854e-587a9d8c0b3e-kube-api-access-4gg7t\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:36.966317 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:36.966208    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/760c0efa-fb24-454c-854e-587a9d8c0b3e-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:12:37.625405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:37.625378    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b"
Jul 23 00:12:37.625405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:37.625396    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540szw2b" event={"ID":"760c0efa-fb24-454c-854e-587a9d8c0b3e","Type":"ContainerDied","Data":"257d95f9a2b18356934d9d4967ed78f61ee4ff408037ddd0958c640e52092652"}
Jul 23 00:12:37.625643 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:37.625436    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="257d95f9a2b18356934d9d4967ed78f61ee4ff408037ddd0958c640e52092652"
Jul 23 00:12:42.774717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.774686    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"]
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775002    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="pull"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775015    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="pull"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775023    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="extract"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775029    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="extract"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775039    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62b2dca4-4a6b-42ef-9254-cb5f7f661295" containerName="console"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775043    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="62b2dca4-4a6b-42ef-9254-cb5f7f661295" containerName="console"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775057    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="util"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775062    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="util"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775127    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="62b2dca4-4a6b-42ef-9254-cb5f7f661295" containerName="console"
Jul 23 00:12:42.775220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.775141    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="760c0efa-fb24-454c-854e-587a9d8c0b3e" containerName="extract"
Jul 23 00:12:42.779415 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.779396    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:42.781724 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.781704    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-94c7m\""
Jul 23 00:12:42.781828 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.781705    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:12:42.781828 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.781797    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 23 00:12:42.789005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.788981    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"]
Jul 23 00:12:42.803048 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.803028    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e160297-5da9-4354-9f4b-58704f911c76-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5762j\" (UID: \"3e160297-5da9-4354-9f4b-58704f911c76\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:42.803187 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.803168    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4www\" (UniqueName: \"kubernetes.io/projected/3e160297-5da9-4354-9f4b-58704f911c76-kube-api-access-t4www\") pod \"cert-manager-operator-controller-manager-55c444bf86-5762j\" (UID: \"3e160297-5da9-4354-9f4b-58704f911c76\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:42.903449 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.903418    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4www\" (UniqueName: \"kubernetes.io/projected/3e160297-5da9-4354-9f4b-58704f911c76-kube-api-access-t4www\") pod \"cert-manager-operator-controller-manager-55c444bf86-5762j\" (UID: \"3e160297-5da9-4354-9f4b-58704f911c76\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:42.903554 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.903455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e160297-5da9-4354-9f4b-58704f911c76-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5762j\" (UID: \"3e160297-5da9-4354-9f4b-58704f911c76\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:42.903763 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.903748    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3e160297-5da9-4354-9f4b-58704f911c76-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-5762j\" (UID: \"3e160297-5da9-4354-9f4b-58704f911c76\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:42.913217 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:42.913193    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4www\" (UniqueName: \"kubernetes.io/projected/3e160297-5da9-4354-9f4b-58704f911c76-kube-api-access-t4www\") pod \"cert-manager-operator-controller-manager-55c444bf86-5762j\" (UID: \"3e160297-5da9-4354-9f4b-58704f911c76\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:43.089682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:43.089658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"
Jul 23 00:12:43.222038 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:43.222016    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j"]
Jul 23 00:12:43.224436 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:12:43.224410    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e160297_5da9_4354_9f4b_58704f911c76.slice/crio-6f6f5ff828246a0ed4d91698a674db362c8f9f0c390a9256ede190caa24ebc3d WatchSource:0}: Error finding container 6f6f5ff828246a0ed4d91698a674db362c8f9f0c390a9256ede190caa24ebc3d: Status 404 returned error can't find the container with id 6f6f5ff828246a0ed4d91698a674db362c8f9f0c390a9256ede190caa24ebc3d
Jul 23 00:12:43.653173 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:43.653142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j" event={"ID":"3e160297-5da9-4354-9f4b-58704f911c76","Type":"ContainerStarted","Data":"6f6f5ff828246a0ed4d91698a674db362c8f9f0c390a9256ede190caa24ebc3d"}
Jul 23 00:12:45.664428 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:45.664392    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j" event={"ID":"3e160297-5da9-4354-9f4b-58704f911c76","Type":"ContainerStarted","Data":"e407f728aa617d5cc350d4d0594f90e861305b5340a05e1f035d017b5f2c3c5c"}
Jul 23 00:12:45.684492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:45.684440    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-5762j" podStartSLOduration=2.050017186 podStartE2EDuration="3.684424507s" totalImagesPullingTime="1.634407321s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:12:45.683282021 +0000 UTC m=+515.407597204" watchObservedRunningTime="2026-07-23 00:12:45.684424507 +0000 UTC m=+515.408739714"
Jul 23 00:12:46.713950 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.713922    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7ckln"]
Jul 23 00:12:46.717204 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.717188    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:46.719741 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.719718    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 23 00:12:46.719885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.719718    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 23 00:12:46.719885 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.719801    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-wfsbg\""
Jul 23 00:12:46.729351 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.729326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmw99\" (UniqueName: \"kubernetes.io/projected/5b40cd19-da22-4b88-a715-9091212508c2-kube-api-access-mmw99\") pod \"cert-manager-webhook-6486ffd997-7ckln\" (UID: \"5b40cd19-da22-4b88-a715-9091212508c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:46.729454 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.729378    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b40cd19-da22-4b88-a715-9091212508c2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7ckln\" (UID: \"5b40cd19-da22-4b88-a715-9091212508c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:46.734650 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.734627    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7ckln"]
Jul 23 00:12:46.830150 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.830120    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmw99\" (UniqueName: \"kubernetes.io/projected/5b40cd19-da22-4b88-a715-9091212508c2-kube-api-access-mmw99\") pod \"cert-manager-webhook-6486ffd997-7ckln\" (UID: \"5b40cd19-da22-4b88-a715-9091212508c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:46.830288 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.830187    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b40cd19-da22-4b88-a715-9091212508c2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7ckln\" (UID: \"5b40cd19-da22-4b88-a715-9091212508c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:46.842350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.842322    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5b40cd19-da22-4b88-a715-9091212508c2-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-7ckln\" (UID: \"5b40cd19-da22-4b88-a715-9091212508c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:46.842461 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:46.842367    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmw99\" (UniqueName: \"kubernetes.io/projected/5b40cd19-da22-4b88-a715-9091212508c2-kube-api-access-mmw99\") pod \"cert-manager-webhook-6486ffd997-7ckln\" (UID: \"5b40cd19-da22-4b88-a715-9091212508c2\") " pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:47.038999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:47.038915    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:47.166682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:47.166628    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-7ckln"]
Jul 23 00:12:47.168874 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:12:47.168831    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5b40cd19_da22_4b88_a715_9091212508c2.slice/crio-3b7360c5ecaaf1b379572cdac671615dfc3bbf997cd482b97620d55f72e6c446 WatchSource:0}: Error finding container 3b7360c5ecaaf1b379572cdac671615dfc3bbf997cd482b97620d55f72e6c446: Status 404 returned error can't find the container with id 3b7360c5ecaaf1b379572cdac671615dfc3bbf997cd482b97620d55f72e6c446
Jul 23 00:12:47.675263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:47.675226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln" event={"ID":"5b40cd19-da22-4b88-a715-9091212508c2","Type":"ContainerStarted","Data":"3b7360c5ecaaf1b379572cdac671615dfc3bbf997cd482b97620d55f72e6c446"}
Jul 23 00:12:50.694069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:50.694037    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln" event={"ID":"5b40cd19-da22-4b88-a715-9091212508c2","Type":"ContainerStarted","Data":"b5a33dad8b7072cd0dc5e16abbe92dddb89af92e5bcd52c02b7a1c1beee541cf"}
Jul 23 00:12:50.694456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:50.694086    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:12:50.713365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:50.713302    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln" podStartSLOduration=1.8249472070000001 podStartE2EDuration="4.713277262s" totalImagesPullingTime="2.888330055s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:12:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:12:50.711379517 +0000 UTC m=+520.435694698" watchObservedRunningTime="2026-07-23 00:12:50.713277262 +0000 UTC m=+520.437592447"
Jul 23 00:12:51.750534 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.750502    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"]
Jul 23 00:12:51.753744 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.753727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:51.759853 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.759822    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-q8x57\""
Jul 23 00:12:51.767113 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.767088    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtvbq\" (UniqueName: \"kubernetes.io/projected/95b686b7-5df1-47f3-9e73-dc9da25fa97d-kube-api-access-qtvbq\") pod \"cert-manager-cainjector-559cdd8c87-p99v6\" (UID: \"95b686b7-5df1-47f3-9e73-dc9da25fa97d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:51.767300 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.767279    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/95b686b7-5df1-47f3-9e73-dc9da25fa97d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p99v6\" (UID: \"95b686b7-5df1-47f3-9e73-dc9da25fa97d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:51.769436 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.769412    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"]
Jul 23 00:12:51.868736 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.868703    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/95b686b7-5df1-47f3-9e73-dc9da25fa97d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p99v6\" (UID: \"95b686b7-5df1-47f3-9e73-dc9da25fa97d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:51.868928 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.868770    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtvbq\" (UniqueName: \"kubernetes.io/projected/95b686b7-5df1-47f3-9e73-dc9da25fa97d-kube-api-access-qtvbq\") pod \"cert-manager-cainjector-559cdd8c87-p99v6\" (UID: \"95b686b7-5df1-47f3-9e73-dc9da25fa97d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:51.876495 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.876470    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/95b686b7-5df1-47f3-9e73-dc9da25fa97d-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-p99v6\" (UID: \"95b686b7-5df1-47f3-9e73-dc9da25fa97d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:51.876596 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:51.876521    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtvbq\" (UniqueName: \"kubernetes.io/projected/95b686b7-5df1-47f3-9e73-dc9da25fa97d-kube-api-access-qtvbq\") pod \"cert-manager-cainjector-559cdd8c87-p99v6\" (UID: \"95b686b7-5df1-47f3-9e73-dc9da25fa97d\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:52.062539 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:52.062516    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"
Jul 23 00:12:52.188549 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:52.188524    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-p99v6"]
Jul 23 00:12:52.191004 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:12:52.190977    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95b686b7_5df1_47f3_9e73_dc9da25fa97d.slice/crio-7f08c3b4a6abb6610c74ad9aa744c5ba257b47a408ba8c2be4259ba323bafb0b WatchSource:0}: Error finding container 7f08c3b4a6abb6610c74ad9aa744c5ba257b47a408ba8c2be4259ba323bafb0b: Status 404 returned error can't find the container with id 7f08c3b4a6abb6610c74ad9aa744c5ba257b47a408ba8c2be4259ba323bafb0b
Jul 23 00:12:52.704785 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:52.704740    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6" event={"ID":"95b686b7-5df1-47f3-9e73-dc9da25fa97d","Type":"ContainerStarted","Data":"1f036c50404ab4b5d3f0ed42c1cd124fb6ea232a9f49c4ed36286b90805bd650"}
Jul 23 00:12:52.705001 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:52.704792    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6" event={"ID":"95b686b7-5df1-47f3-9e73-dc9da25fa97d","Type":"ContainerStarted","Data":"7f08c3b4a6abb6610c74ad9aa744c5ba257b47a408ba8c2be4259ba323bafb0b"}
Jul 23 00:12:52.721234 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:52.721187    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-p99v6" podStartSLOduration=1.721171736 podStartE2EDuration="1.721171736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:12:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:12:52.719400465 +0000 UTC m=+522.443715686" watchObservedRunningTime="2026-07-23 00:12:52.721171736 +0000 UTC m=+522.445486919"
Jul 23 00:12:56.700847 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:12:56.700817    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-7ckln"
Jul 23 00:13:10.387699 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.387622    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"]
Jul 23 00:13:10.390108 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.390089    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.392455 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.392429    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 00:13:10.393289 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.393228    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:13:10.393389 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.393316    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4hb9c\""
Jul 23 00:13:10.398977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.398955    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"]
Jul 23 00:13:10.497851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.497806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-954tr\" (UniqueName: \"kubernetes.io/projected/029e47fc-9c1f-44c9-964c-95cd1093b90b-kube-api-access-954tr\") pod \"openshift-lws-operator-fd8ccff4c-72grd\" (UID: \"029e47fc-9c1f-44c9-964c-95cd1093b90b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.497957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.497871    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029e47fc-9c1f-44c9-964c-95cd1093b90b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-72grd\" (UID: \"029e47fc-9c1f-44c9-964c-95cd1093b90b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.599059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.599036    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-954tr\" (UniqueName: \"kubernetes.io/projected/029e47fc-9c1f-44c9-964c-95cd1093b90b-kube-api-access-954tr\") pod \"openshift-lws-operator-fd8ccff4c-72grd\" (UID: \"029e47fc-9c1f-44c9-964c-95cd1093b90b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.599168 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.599078    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029e47fc-9c1f-44c9-964c-95cd1093b90b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-72grd\" (UID: \"029e47fc-9c1f-44c9-964c-95cd1093b90b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.599390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.599375    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/029e47fc-9c1f-44c9-964c-95cd1093b90b-tmp\") pod \"openshift-lws-operator-fd8ccff4c-72grd\" (UID: \"029e47fc-9c1f-44c9-964c-95cd1093b90b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.605894 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.605872    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 23 00:13:10.617786 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.617758    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 23 00:13:10.628897 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.628876    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-954tr\" (UniqueName: \"kubernetes.io/projected/029e47fc-9c1f-44c9-964c-95cd1093b90b-kube-api-access-954tr\") pod \"openshift-lws-operator-fd8ccff4c-72grd\" (UID: \"029e47fc-9c1f-44c9-964c-95cd1093b90b\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.703186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.703167    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-4hb9c\""
Jul 23 00:13:10.711354 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.711336    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"
Jul 23 00:13:10.838152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:10.838129    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd"]
Jul 23 00:13:10.840338 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:13:10.840307    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod029e47fc_9c1f_44c9_964c_95cd1093b90b.slice/crio-6b9181dc2a1ef713afff815a44190aad8e7553de458c0502beb67c6cc87b7d9c WatchSource:0}: Error finding container 6b9181dc2a1ef713afff815a44190aad8e7553de458c0502beb67c6cc87b7d9c: Status 404 returned error can't find the container with id 6b9181dc2a1ef713afff815a44190aad8e7553de458c0502beb67c6cc87b7d9c
Jul 23 00:13:11.795675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:11.795633    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd" event={"ID":"029e47fc-9c1f-44c9-964c-95cd1093b90b","Type":"ContainerStarted","Data":"6b9181dc2a1ef713afff815a44190aad8e7553de458c0502beb67c6cc87b7d9c"}
Jul 23 00:13:13.807673 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:13.807644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd" event={"ID":"029e47fc-9c1f-44c9-964c-95cd1093b90b","Type":"ContainerStarted","Data":"2f0120a2c54fbaf54ff67c85dd98f5cbda1662b67b1269b6953fd4caf4ead687"}
Jul 23 00:13:13.822212 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:13.822131    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-72grd" podStartSLOduration=1.091812006 podStartE2EDuration="3.822113091s" totalImagesPullingTime="2.730301085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:13:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:13:13.821597698 +0000 UTC m=+543.545912881" watchObservedRunningTime="2026-07-23 00:13:13.822113091 +0000 UTC m=+543.546428275"
Jul 23 00:13:32.992430 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:32.992394    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"]
Jul 23 00:13:32.996659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:32.996642    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.000509 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.000486    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 23 00:13:33.000640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.000507    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mjztj\""
Jul 23 00:13:33.000640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.000576    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 23 00:13:33.000640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.000586    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 23 00:13:33.009368 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.009347    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"]
Jul 23 00:13:33.072356 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.072329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8ddf83f-0651-4f88-afa0-a2348cec97ac-cert\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.072470 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.072366    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgbgz\" (UniqueName: \"kubernetes.io/projected/e8ddf83f-0651-4f88-afa0-a2348cec97ac-kube-api-access-lgbgz\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.072470 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.072390    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8ddf83f-0651-4f88-afa0-a2348cec97ac-metrics-cert\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.072470 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.072435    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8ddf83f-0651-4f88-afa0-a2348cec97ac-manager-config\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.173683 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.173646    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8ddf83f-0651-4f88-afa0-a2348cec97ac-metrics-cert\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.173683 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.173694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8ddf83f-0651-4f88-afa0-a2348cec97ac-manager-config\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.173908 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.173757    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8ddf83f-0651-4f88-afa0-a2348cec97ac-cert\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.173908 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.173777    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgbgz\" (UniqueName: \"kubernetes.io/projected/e8ddf83f-0651-4f88-afa0-a2348cec97ac-kube-api-access-lgbgz\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.174452 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.174432    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e8ddf83f-0651-4f88-afa0-a2348cec97ac-manager-config\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.176350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.176317    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e8ddf83f-0651-4f88-afa0-a2348cec97ac-metrics-cert\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.176442 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.176317    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e8ddf83f-0651-4f88-afa0-a2348cec97ac-cert\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.182528 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.182502    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgbgz\" (UniqueName: \"kubernetes.io/projected/e8ddf83f-0651-4f88-afa0-a2348cec97ac-kube-api-access-lgbgz\") pod \"lws-controller-manager-7dcccd7c7c-49mfj\" (UID: \"e8ddf83f-0651-4f88-afa0-a2348cec97ac\") " pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.306177 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.306116    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:33.449385 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.449363    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"]
Jul 23 00:13:33.451604 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:13:33.451576    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8ddf83f_0651_4f88_afa0_a2348cec97ac.slice/crio-98234fcbcb9d3f405b5360738e134c99ea9c83d25ce5c6fc6a64ef9b3790e336 WatchSource:0}: Error finding container 98234fcbcb9d3f405b5360738e134c99ea9c83d25ce5c6fc6a64ef9b3790e336: Status 404 returned error can't find the container with id 98234fcbcb9d3f405b5360738e134c99ea9c83d25ce5c6fc6a64ef9b3790e336
Jul 23 00:13:33.897891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:33.897831    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj" event={"ID":"e8ddf83f-0651-4f88-afa0-a2348cec97ac","Type":"ContainerStarted","Data":"98234fcbcb9d3f405b5360738e134c99ea9c83d25ce5c6fc6a64ef9b3790e336"}
Jul 23 00:13:35.910727 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:35.910680    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj" event={"ID":"e8ddf83f-0651-4f88-afa0-a2348cec97ac","Type":"ContainerStarted","Data":"c25abb9aefbdaa9d5e28842f6f45b3014a9d8ccb07819f06721ab34546f103ae"}
Jul 23 00:13:35.911219 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:35.910748    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:35.931305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:35.931252    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj" podStartSLOduration=2.305371933 podStartE2EDuration="3.931236835s" totalImagesPullingTime="1.625864902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:13:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:13:35.930028187 +0000 UTC m=+565.654343393" watchObservedRunningTime="2026-07-23 00:13:35.931236835 +0000 UTC m=+565.655552018"
Jul 23 00:13:42.529045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.529011    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"]
Jul 23 00:13:42.532667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.532651    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.539545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.539516    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 23 00:13:42.539657 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.539559    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 23 00:13:42.539711 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.539657    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 23 00:13:42.539870 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.539850    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 23 00:13:42.539942 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.539894    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-zvmmz\""
Jul 23 00:13:42.579954 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.579932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"]
Jul 23 00:13:42.648869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.648821    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-tmp\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.648982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.648949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-manifests\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.648982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.648972    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.649116 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.649007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-webhook-cert\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.649116 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.649039    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgf55\" (UniqueName: \"kubernetes.io/projected/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-kube-api-access-rgf55\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.750061 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.750034    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-manifests\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.750061 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.750063    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.750242 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.750080    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-webhook-cert\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.750242 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.750107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgf55\" (UniqueName: \"kubernetes.io/projected/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-kube-api-access-rgf55\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.750242 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.750141    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-tmp\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.750403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.750391    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-manifests\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.752394 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.752368    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-tmp\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.752483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.752441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.752595 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.752580    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-webhook-cert\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.759701 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.759677    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgf55\" (UniqueName: \"kubernetes.io/projected/2ed5c242-d656-4ccc-a10a-26a9d7538ae5-kube-api-access-rgf55\") pod \"opendatahub-operator-controller-manager-86b98fc76f-hdt4v\" (UID: \"2ed5c242-d656-4ccc-a10a-26a9d7538ae5\") " pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.843421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.843400    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:42.976086 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:42.976063    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"]
Jul 23 00:13:42.978105 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:13:42.978077    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ed5c242_d656_4ccc_a10a_26a9d7538ae5.slice/crio-772e252a2e2c8278985169927ac37967a61748f74c472a6c74aa2b5658afe2ab WatchSource:0}: Error finding container 772e252a2e2c8278985169927ac37967a61748f74c472a6c74aa2b5658afe2ab: Status 404 returned error can't find the container with id 772e252a2e2c8278985169927ac37967a61748f74c472a6c74aa2b5658afe2ab
Jul 23 00:13:43.952044 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:43.952008    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v" event={"ID":"2ed5c242-d656-4ccc-a10a-26a9d7538ae5","Type":"ContainerStarted","Data":"772e252a2e2c8278985169927ac37967a61748f74c472a6c74aa2b5658afe2ab"}
Jul 23 00:13:46.917614 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:46.917584    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7dcccd7c7c-49mfj"
Jul 23 00:13:46.969571 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:46.969543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v" event={"ID":"2ed5c242-d656-4ccc-a10a-26a9d7538ae5","Type":"ContainerStarted","Data":"016ab19db26145c355ecf365b9ff2fa696b3e73123940a579329094bf5759e1f"}
Jul 23 00:13:47.975485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:47.975449    2558 generic.go:356] "Generic (PLEG): container finished" podID="2ed5c242-d656-4ccc-a10a-26a9d7538ae5" containerID="016ab19db26145c355ecf365b9ff2fa696b3e73123940a579329094bf5759e1f" exitCode=0
Jul 23 00:13:47.976061 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:47.975538    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v" event={"ID":"2ed5c242-d656-4ccc-a10a-26a9d7538ae5","Type":"ContainerDied","Data":"016ab19db26145c355ecf365b9ff2fa696b3e73123940a579329094bf5759e1f"}
Jul 23 00:13:48.982514 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:48.982483    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v" event={"ID":"2ed5c242-d656-4ccc-a10a-26a9d7538ae5","Type":"ContainerStarted","Data":"f90a5813b2ac7a4a6bde69870c20d33593bc4838df0e7f639261fbd94e9c891f"}
Jul 23 00:13:48.982900 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:48.982611    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:13:49.006966 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:49.006908    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v" podStartSLOduration=2.4843538929999998 podStartE2EDuration="7.006889272s" totalImagesPullingTime="3.457823272s" totalInitContainerRuntime="1.064712107s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:13:42 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:13:49.005492807 +0000 UTC m=+578.729807991" watchObservedRunningTime="2026-07-23 00:13:49.006889272 +0000 UTC m=+578.731204490"
Jul 23 00:13:59.990544 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:13:59.990516    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-86b98fc76f-hdt4v"
Jul 23 00:14:10.665932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:10.665906    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:14:10.666342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:10.666293    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:14:20.468690 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.468659    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"]
Jul 23 00:14:20.478938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.478917    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.481605 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.481398    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 23 00:14:20.481605 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.481484    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-5qnf9\""
Jul 23 00:14:20.482883 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.482859    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"]
Jul 23 00:14:20.544280 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544251    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544303    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544349    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxjkd\" (UniqueName: \"kubernetes.io/projected/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-kube-api-access-fxjkd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544385    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544416    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544454    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544487    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544547    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.544611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.544590    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645217 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645185    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645228    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645253    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxjkd\" (UniqueName: \"kubernetes.io/projected/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-kube-api-access-fxjkd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645278    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645391    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645452    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645524    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645555    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645621    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645749 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645660    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645777    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.645932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.645906    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.646237 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.646216    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.647706 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.647686    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.647795 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.647751    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.653081 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.653055    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.653227 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.653209    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxjkd\" (UniqueName: \"kubernetes.io/projected/cf4c1ae8-bd08-46ff-8c1e-71a264c49963-kube-api-access-fxjkd\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4wgsm4\" (UID: \"cf4c1ae8-bd08-46ff-8c1e-71a264c49963\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.790477 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.790395    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:20.932408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:20.932380    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"]
Jul 23 00:14:20.937546 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:14:20.937516    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf4c1ae8_bd08_46ff_8c1e_71a264c49963.slice/crio-4d3e27a9b491b4e03bc58f23bde1da61dfd59d2cb7d84520945faa368b0a45d3 WatchSource:0}: Error finding container 4d3e27a9b491b4e03bc58f23bde1da61dfd59d2cb7d84520945faa368b0a45d3: Status 404 returned error can't find the container with id 4d3e27a9b491b4e03bc58f23bde1da61dfd59d2cb7d84520945faa368b0a45d3
Jul 23 00:14:21.142649 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:21.142616    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4" event={"ID":"cf4c1ae8-bd08-46ff-8c1e-71a264c49963","Type":"ContainerStarted","Data":"4d3e27a9b491b4e03bc58f23bde1da61dfd59d2cb7d84520945faa368b0a45d3"}
Jul 23 00:14:24.072527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:24.072494    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:14:24.072771 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:24.072564    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:14:24.072771 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:24.072592    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:14:24.168712 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:24.168678    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4" event={"ID":"cf4c1ae8-bd08-46ff-8c1e-71a264c49963","Type":"ContainerStarted","Data":"79a11b63b3c43dfeeae424314cc1d9b6e81c19ba5884781a3009fd96440990f9"}
Jul 23 00:14:24.189219 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:24.189170    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4" podStartSLOduration=1.057566358 podStartE2EDuration="4.189155777s" totalImagesPullingTime="3.131589419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:14:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:14:24.18568797 +0000 UTC m=+613.910003154" watchObservedRunningTime="2026-07-23 00:14:24.189155777 +0000 UTC m=+613.913470960"
Jul 23 00:14:24.790925 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:24.790891    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:25.795204 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:25.795173    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:26.177760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:26.177726    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:14:26.178474 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:14:26.178456    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4wgsm4"
Jul 23 00:15:03.588769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.588694    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnx67"]
Jul 23 00:15:03.604581 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.604546    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:03.606802 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.606773    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnx67"]
Jul 23 00:15:03.607336 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.607315    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-bbzd2\""
Jul 23 00:15:03.607518 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.607355    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 23 00:15:03.607634 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.607439    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 23 00:15:03.783879 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.783855    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dd9r4\" (UniqueName: \"kubernetes.io/projected/280d2d13-d940-48b1-b0ef-dc7aa84ad852-kube-api-access-dd9r4\") pod \"kuadrant-operator-catalog-jnx67\" (UID: \"280d2d13-d940-48b1-b0ef-dc7aa84ad852\") " pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:03.884979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.884917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dd9r4\" (UniqueName: \"kubernetes.io/projected/280d2d13-d940-48b1-b0ef-dc7aa84ad852-kube-api-access-dd9r4\") pod \"kuadrant-operator-catalog-jnx67\" (UID: \"280d2d13-d940-48b1-b0ef-dc7aa84ad852\") " pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:03.893145 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.893116    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dd9r4\" (UniqueName: \"kubernetes.io/projected/280d2d13-d940-48b1-b0ef-dc7aa84ad852-kube-api-access-dd9r4\") pod \"kuadrant-operator-catalog-jnx67\" (UID: \"280d2d13-d940-48b1-b0ef-dc7aa84ad852\") " pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:03.915904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.915882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:03.958482 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:03.958454    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnx67"]
Jul 23 00:15:04.041472 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.041445    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnx67"]
Jul 23 00:15:04.043364 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:04.043334    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280d2d13_d940_48b1_b0ef_dc7aa84ad852.slice/crio-2687b725f4942a44488abf6f131a260fa1c2016194321298dd5903b99bfcbf1e WatchSource:0}: Error finding container 2687b725f4942a44488abf6f131a260fa1c2016194321298dd5903b99bfcbf1e: Status 404 returned error can't find the container with id 2687b725f4942a44488abf6f131a260fa1c2016194321298dd5903b99bfcbf1e
Jul 23 00:15:04.167780 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.167725    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lzvg7"]
Jul 23 00:15:04.172685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.172670    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:04.177982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.177963    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lzvg7"]
Jul 23 00:15:04.288235 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.288208    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd7v4\" (UniqueName: \"kubernetes.io/projected/eda5c88d-76ed-4cbe-bf15-2e80aa228982-kube-api-access-qd7v4\") pod \"kuadrant-operator-catalog-lzvg7\" (UID: \"eda5c88d-76ed-4cbe-bf15-2e80aa228982\") " pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:04.363011 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.362977    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnx67" event={"ID":"280d2d13-d940-48b1-b0ef-dc7aa84ad852","Type":"ContainerStarted","Data":"2687b725f4942a44488abf6f131a260fa1c2016194321298dd5903b99bfcbf1e"}
Jul 23 00:15:04.389457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.389434    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd7v4\" (UniqueName: \"kubernetes.io/projected/eda5c88d-76ed-4cbe-bf15-2e80aa228982-kube-api-access-qd7v4\") pod \"kuadrant-operator-catalog-lzvg7\" (UID: \"eda5c88d-76ed-4cbe-bf15-2e80aa228982\") " pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:04.397911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.397893    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd7v4\" (UniqueName: \"kubernetes.io/projected/eda5c88d-76ed-4cbe-bf15-2e80aa228982-kube-api-access-qd7v4\") pod \"kuadrant-operator-catalog-lzvg7\" (UID: \"eda5c88d-76ed-4cbe-bf15-2e80aa228982\") " pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:04.482329 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.482266    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:04.609622 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:04.609591    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-lzvg7"]
Jul 23 00:15:04.612628 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:04.612602    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeda5c88d_76ed_4cbe_bf15_2e80aa228982.slice/crio-f5f5f7cf5ab265dff50b4adca6602620ae317a7c57555b9dec9734033db45c3d WatchSource:0}: Error finding container f5f5f7cf5ab265dff50b4adca6602620ae317a7c57555b9dec9734033db45c3d: Status 404 returned error can't find the container with id f5f5f7cf5ab265dff50b4adca6602620ae317a7c57555b9dec9734033db45c3d
Jul 23 00:15:05.369670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:05.369634    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7" event={"ID":"eda5c88d-76ed-4cbe-bf15-2e80aa228982","Type":"ContainerStarted","Data":"f5f5f7cf5ab265dff50b4adca6602620ae317a7c57555b9dec9734033db45c3d"}
Jul 23 00:15:07.384067 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.384030    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7" event={"ID":"eda5c88d-76ed-4cbe-bf15-2e80aa228982","Type":"ContainerStarted","Data":"88e6432c16f59018a2ee4731bf3eb45e1f79708bfd15fb42610e36ecdd5dd375"}
Jul 23 00:15:07.385453 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.385430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnx67" event={"ID":"280d2d13-d940-48b1-b0ef-dc7aa84ad852","Type":"ContainerStarted","Data":"d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823"}
Jul 23 00:15:07.385566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.385518    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jnx67" podUID="280d2d13-d940-48b1-b0ef-dc7aa84ad852" containerName="registry-server" containerID="cri-o://d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823" gracePeriod=2
Jul 23 00:15:07.397716 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.397677    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7" podStartSLOduration=1.517312708 podStartE2EDuration="3.397664917s" totalImagesPullingTime="1.880352209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:15:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:15:07.397042476 +0000 UTC m=+657.121357685" watchObservedRunningTime="2026-07-23 00:15:07.397664917 +0000 UTC m=+657.121980079"
Jul 23 00:15:07.411552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.411516    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jnx67" podStartSLOduration=1.9793368980000001 podStartE2EDuration="4.411505379s" totalImagesPullingTime="2.432168481s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:15:07.410400119 +0000 UTC m=+657.134715306" watchObservedRunningTime="2026-07-23 00:15:07.411505379 +0000 UTC m=+657.135820564"
Jul 23 00:15:07.630555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.630528    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:07.721145 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.721080    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dd9r4\" (UniqueName: \"kubernetes.io/projected/280d2d13-d940-48b1-b0ef-dc7aa84ad852-kube-api-access-dd9r4\") pod \"280d2d13-d940-48b1-b0ef-dc7aa84ad852\" (UID: \"280d2d13-d940-48b1-b0ef-dc7aa84ad852\") "
Jul 23 00:15:07.723237 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.723207    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/280d2d13-d940-48b1-b0ef-dc7aa84ad852-kube-api-access-dd9r4" (OuterVolumeSpecName: "kube-api-access-dd9r4") pod "280d2d13-d940-48b1-b0ef-dc7aa84ad852" (UID: "280d2d13-d940-48b1-b0ef-dc7aa84ad852"). InnerVolumeSpecName "kube-api-access-dd9r4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:15:07.822056 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:07.822020    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dd9r4\" (UniqueName: \"kubernetes.io/projected/280d2d13-d940-48b1-b0ef-dc7aa84ad852-kube-api-access-dd9r4\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:08.391934 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.391901    2558 generic.go:356] "Generic (PLEG): container finished" podID="280d2d13-d940-48b1-b0ef-dc7aa84ad852" containerID="d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823" exitCode=0
Jul 23 00:15:08.392382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.391957    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jnx67"
Jul 23 00:15:08.392382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.391990    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnx67" event={"ID":"280d2d13-d940-48b1-b0ef-dc7aa84ad852","Type":"ContainerDied","Data":"d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823"}
Jul 23 00:15:08.392382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.392085    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jnx67" event={"ID":"280d2d13-d940-48b1-b0ef-dc7aa84ad852","Type":"ContainerDied","Data":"2687b725f4942a44488abf6f131a260fa1c2016194321298dd5903b99bfcbf1e"}
Jul 23 00:15:08.392382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.392116    2558 scope.go:117] "RemoveContainer" containerID="d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823"
Jul 23 00:15:08.401314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.401298    2558 scope.go:117] "RemoveContainer" containerID="d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823"
Jul 23 00:15:08.401576 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:15:08.401555    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823\": container with ID starting with d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823 not found: ID does not exist" containerID="d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823"
Jul 23 00:15:08.401641 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.401583    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823"} err="failed to get container status \"d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823\": rpc error: code = NotFound desc = could not find container \"d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823\": container with ID starting with d35f14d4f612f5fdbcc2d91fa88c7c316718353b8ad354886e8c84be35022823 not found: ID does not exist"
Jul 23 00:15:08.413055 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.413026    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnx67"]
Jul 23 00:15:08.414762 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.414737    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jnx67"]
Jul 23 00:15:08.735287 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:08.735223    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="280d2d13-d940-48b1-b0ef-dc7aa84ad852" path="/var/lib/kubelet/pods/280d2d13-d940-48b1-b0ef-dc7aa84ad852/volumes"
Jul 23 00:15:14.482675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:14.482592    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:14.482675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:14.482669    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:14.504893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:14.504870    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:15.453576 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:15.453538    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-lzvg7"
Jul 23 00:15:19.202534 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.202498    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"]
Jul 23 00:15:19.202955 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.202907    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="280d2d13-d940-48b1-b0ef-dc7aa84ad852" containerName="registry-server"
Jul 23 00:15:19.202955 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.202920    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="280d2d13-d940-48b1-b0ef-dc7aa84ad852" containerName="registry-server"
Jul 23 00:15:19.203041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.202986    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="280d2d13-d940-48b1-b0ef-dc7aa84ad852" containerName="registry-server"
Jul 23 00:15:19.207620 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.207601    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.210284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.210259    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mrkb7\""
Jul 23 00:15:19.213251 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.213219    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"]
Jul 23 00:15:19.308655 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.308631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.308765 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.308659    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2btjn\" (UniqueName: \"kubernetes.io/projected/d729d699-83ae-42ee-8c4a-7fafe7e59d90-kube-api-access-2btjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.308765 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.308683    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.409527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.409501    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.409611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.409531    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2btjn\" (UniqueName: \"kubernetes.io/projected/d729d699-83ae-42ee-8c4a-7fafe7e59d90-kube-api-access-2btjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.409611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.409557    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.409934 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.409918    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.409995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.409977    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.420196 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.420163    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2btjn\" (UniqueName: \"kubernetes.io/projected/d729d699-83ae-42ee-8c4a-7fafe7e59d90-kube-api-access-2btjn\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.518044 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.517996    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:19.598612 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.598584    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"]
Jul 23 00:15:19.603806 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.603784    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.609030 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.609004    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"]
Jul 23 00:15:19.611308 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.611281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.611436 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.611333    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw6ln\" (UniqueName: \"kubernetes.io/projected/ca48f794-daa3-41cf-9f07-56fb81211503-kube-api-access-sw6ln\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.611500 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.611486    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.650698 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.650677    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"]
Jul 23 00:15:19.653063 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:19.653040    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd729d699_83ae_42ee_8c4a_7fafe7e59d90.slice/crio-4e89db3e501d0b7e9226647cc2f79d10671b58d4f86d476913ee7a406aaff379 WatchSource:0}: Error finding container 4e89db3e501d0b7e9226647cc2f79d10671b58d4f86d476913ee7a406aaff379: Status 404 returned error can't find the container with id 4e89db3e501d0b7e9226647cc2f79d10671b58d4f86d476913ee7a406aaff379
Jul 23 00:15:19.712342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.712317    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.712454 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.712353    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.712454 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.712383    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw6ln\" (UniqueName: \"kubernetes.io/projected/ca48f794-daa3-41cf-9f07-56fb81211503-kube-api-access-sw6ln\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.712679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.712656    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.712735 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.712711    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.719539 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.719515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw6ln\" (UniqueName: \"kubernetes.io/projected/ca48f794-daa3-41cf-9f07-56fb81211503-kube-api-access-sw6ln\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:19.917280 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:19.917254    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:20.001627 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.001597    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"]
Jul 23 00:15:20.007316 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.007293    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.013238 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.013174    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"]
Jul 23 00:15:20.015530 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.015445    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.015698 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.015558    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.015698 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.015639    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntgk5\" (UniqueName: \"kubernetes.io/projected/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-kube-api-access-ntgk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.051146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.051124    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"]
Jul 23 00:15:20.052643 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:20.052619    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca48f794_daa3_41cf_9f07_56fb81211503.slice/crio-0fe758ced0bcb0e001dec43c336b54792a9307a2696607491422c3f839ef51ff WatchSource:0}: Error finding container 0fe758ced0bcb0e001dec43c336b54792a9307a2696607491422c3f839ef51ff: Status 404 returned error can't find the container with id 0fe758ced0bcb0e001dec43c336b54792a9307a2696607491422c3f839ef51ff
Jul 23 00:15:20.116936 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.116913    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.117039 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.117000    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntgk5\" (UniqueName: \"kubernetes.io/projected/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-kube-api-access-ntgk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.117102 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.117085    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.117226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.117205    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.117401 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.117382    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.123984 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.123956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntgk5\" (UniqueName: \"kubernetes.io/projected/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-kube-api-access-ntgk5\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.322911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.322883    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:20.403006 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.402680    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"]
Jul 23 00:15:20.408153 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.408128    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.415851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.415815    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"]
Jul 23 00:15:20.419573 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.419544    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.419667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.419594    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2hpk\" (UniqueName: \"kubernetes.io/projected/d1a95112-5955-4326-9e24-2ef5e1164095-kube-api-access-k2hpk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.419734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.419716    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.453384 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.453348    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"]
Jul 23 00:15:20.455584 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:20.455560    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf12feec1_0eb2_4580_9ce4_1f8a0ab96826.slice/crio-7b4ae4f866e6318077bf038b4f9b1761bb00963d432de7c4dc492549659cfee9 WatchSource:0}: Error finding container 7b4ae4f866e6318077bf038b4f9b1761bb00963d432de7c4dc492549659cfee9: Status 404 returned error can't find the container with id 7b4ae4f866e6318077bf038b4f9b1761bb00963d432de7c4dc492549659cfee9
Jul 23 00:15:20.457244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.457220    2558 generic.go:356] "Generic (PLEG): container finished" podID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerID="707c20ddf3867c5b5eba108c7e81ef2019926f79c682b3d4c5763b6ceee4ced1" exitCode=0
Jul 23 00:15:20.457349 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.457250    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8" event={"ID":"d729d699-83ae-42ee-8c4a-7fafe7e59d90","Type":"ContainerDied","Data":"707c20ddf3867c5b5eba108c7e81ef2019926f79c682b3d4c5763b6ceee4ced1"}
Jul 23 00:15:20.457349 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.457301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8" event={"ID":"d729d699-83ae-42ee-8c4a-7fafe7e59d90","Type":"ContainerStarted","Data":"4e89db3e501d0b7e9226647cc2f79d10671b58d4f86d476913ee7a406aaff379"}
Jul 23 00:15:20.458018 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.457999    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:15:20.459021 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.459000    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca48f794-daa3-41cf-9f07-56fb81211503" containerID="1e02073b552bff2be4f8b2078550831390979b7f3b74613bb286ce0d8324ecb3" exitCode=0
Jul 23 00:15:20.459157 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.459135    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4" event={"ID":"ca48f794-daa3-41cf-9f07-56fb81211503","Type":"ContainerDied","Data":"1e02073b552bff2be4f8b2078550831390979b7f3b74613bb286ce0d8324ecb3"}
Jul 23 00:15:20.459261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.459183    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4" event={"ID":"ca48f794-daa3-41cf-9f07-56fb81211503","Type":"ContainerStarted","Data":"0fe758ced0bcb0e001dec43c336b54792a9307a2696607491422c3f839ef51ff"}
Jul 23 00:15:20.520111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.520088    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.520214 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.520149    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.520214 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.520177    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2hpk\" (UniqueName: \"kubernetes.io/projected/d1a95112-5955-4326-9e24-2ef5e1164095-kube-api-access-k2hpk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.520496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.520477    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.520566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.520545    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.527083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.527063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2hpk\" (UniqueName: \"kubernetes.io/projected/d1a95112-5955-4326-9e24-2ef5e1164095-kube-api-access-k2hpk\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.720208 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.720159    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:20.857649 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:20.857625    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"]
Jul 23 00:15:20.859031 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:20.859010    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a95112_5955_4326_9e24_2ef5e1164095.slice/crio-4f606c06e7361ef35a4728c012bb7b20f47c2f9642c837414cdbc344fe9198e3 WatchSource:0}: Error finding container 4f606c06e7361ef35a4728c012bb7b20f47c2f9642c837414cdbc344fe9198e3: Status 404 returned error can't find the container with id 4f606c06e7361ef35a4728c012bb7b20f47c2f9642c837414cdbc344fe9198e3
Jul 23 00:15:21.465928 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.465857    2558 generic.go:356] "Generic (PLEG): container finished" podID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerID="d1880f4b19d3221fc08047b16bc9354b3ee8869b2291bff7769fb0f2edd916e3" exitCode=0
Jul 23 00:15:21.466252 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.465924    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8" event={"ID":"d729d699-83ae-42ee-8c4a-7fafe7e59d90","Type":"ContainerDied","Data":"d1880f4b19d3221fc08047b16bc9354b3ee8869b2291bff7769fb0f2edd916e3"}
Jul 23 00:15:21.467535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.467494    2558 generic.go:356] "Generic (PLEG): container finished" podID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerID="115e3ba51356f014a8542bd53a759ae5d2d2c502ca885d88546ceac55d7e1878" exitCode=0
Jul 23 00:15:21.467589 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.467575    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh" event={"ID":"f12feec1-0eb2-4580-9ce4-1f8a0ab96826","Type":"ContainerDied","Data":"115e3ba51356f014a8542bd53a759ae5d2d2c502ca885d88546ceac55d7e1878"}
Jul 23 00:15:21.467637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.467610    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh" event={"ID":"f12feec1-0eb2-4580-9ce4-1f8a0ab96826","Type":"ContainerStarted","Data":"7b4ae4f866e6318077bf038b4f9b1761bb00963d432de7c4dc492549659cfee9"}
Jul 23 00:15:21.479223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.479203    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca48f794-daa3-41cf-9f07-56fb81211503" containerID="01911ad453abc6b652690acc7781276f12bab2e679bffc1c4e19ea16d8f7a25a" exitCode=0
Jul 23 00:15:21.479331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.479261    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4" event={"ID":"ca48f794-daa3-41cf-9f07-56fb81211503","Type":"ContainerDied","Data":"01911ad453abc6b652690acc7781276f12bab2e679bffc1c4e19ea16d8f7a25a"}
Jul 23 00:15:21.481171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.481144    2558 generic.go:356] "Generic (PLEG): container finished" podID="d1a95112-5955-4326-9e24-2ef5e1164095" containerID="c3c626773f0fbda2cce52bef6d0b2e612f5e86c8ace0a5402f11ebb344ac9332" exitCode=0
Jul 23 00:15:21.481410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.481196    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw" event={"ID":"d1a95112-5955-4326-9e24-2ef5e1164095","Type":"ContainerDied","Data":"c3c626773f0fbda2cce52bef6d0b2e612f5e86c8ace0a5402f11ebb344ac9332"}
Jul 23 00:15:21.481410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:21.481233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw" event={"ID":"d1a95112-5955-4326-9e24-2ef5e1164095","Type":"ContainerStarted","Data":"4f606c06e7361ef35a4728c012bb7b20f47c2f9642c837414cdbc344fe9198e3"}
Jul 23 00:15:22.489720 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:22.489692    2558 generic.go:356] "Generic (PLEG): container finished" podID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerID="da2a52700dcd26c1370853ae89bb3c330fea7b2a5ef64e0ef1a891aa05176ce2" exitCode=0
Jul 23 00:15:22.490148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:22.489768    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8" event={"ID":"d729d699-83ae-42ee-8c4a-7fafe7e59d90","Type":"ContainerDied","Data":"da2a52700dcd26c1370853ae89bb3c330fea7b2a5ef64e0ef1a891aa05176ce2"}
Jul 23 00:15:22.491570 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:22.491554    2558 generic.go:356] "Generic (PLEG): container finished" podID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerID="ebdaaf35525fb685692e785cac6ce5c208b506d8e526e0aa46c61492911e6aa4" exitCode=0
Jul 23 00:15:22.491662 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:22.491635    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh" event={"ID":"f12feec1-0eb2-4580-9ce4-1f8a0ab96826","Type":"ContainerDied","Data":"ebdaaf35525fb685692e785cac6ce5c208b506d8e526e0aa46c61492911e6aa4"}
Jul 23 00:15:22.493755 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:22.493734    2558 generic.go:356] "Generic (PLEG): container finished" podID="ca48f794-daa3-41cf-9f07-56fb81211503" containerID="fea0fadc92cdf8d3e5767620f2478de7f6ad53e023434406bf52ad9a27d5f938" exitCode=0
Jul 23 00:15:22.493866 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:22.493815    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4" event={"ID":"ca48f794-daa3-41cf-9f07-56fb81211503","Type":"ContainerDied","Data":"fea0fadc92cdf8d3e5767620f2478de7f6ad53e023434406bf52ad9a27d5f938"}
Jul 23 00:15:23.501743 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.501700    2558 generic.go:356] "Generic (PLEG): container finished" podID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerID="4ed81edaa4b1d0042db9d8724987a5c6f8cd82aca5cacadd6226b69951179792" exitCode=0
Jul 23 00:15:23.502148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.501781    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh" event={"ID":"f12feec1-0eb2-4580-9ce4-1f8a0ab96826","Type":"ContainerDied","Data":"4ed81edaa4b1d0042db9d8724987a5c6f8cd82aca5cacadd6226b69951179792"}
Jul 23 00:15:23.503601 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.503578    2558 generic.go:356] "Generic (PLEG): container finished" podID="d1a95112-5955-4326-9e24-2ef5e1164095" containerID="0d74064ed23472596d6929e74876ef3416060167e1c6a588c50e1f8ff6f5523e" exitCode=0
Jul 23 00:15:23.503738 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.503668    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw" event={"ID":"d1a95112-5955-4326-9e24-2ef5e1164095","Type":"ContainerDied","Data":"0d74064ed23472596d6929e74876ef3416060167e1c6a588c50e1f8ff6f5523e"}
Jul 23 00:15:23.650510 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.650484    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:23.734095 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.734074    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:23.747401 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.747373    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw6ln\" (UniqueName: \"kubernetes.io/projected/ca48f794-daa3-41cf-9f07-56fb81211503-kube-api-access-sw6ln\") pod \"ca48f794-daa3-41cf-9f07-56fb81211503\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") "
Jul 23 00:15:23.747505 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.747413    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-util\") pod \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") "
Jul 23 00:15:23.747505 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.747446    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-bundle\") pod \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") "
Jul 23 00:15:23.747505 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.747480    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-util\") pod \"ca48f794-daa3-41cf-9f07-56fb81211503\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") "
Jul 23 00:15:23.747637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.747529    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-bundle\") pod \"ca48f794-daa3-41cf-9f07-56fb81211503\" (UID: \"ca48f794-daa3-41cf-9f07-56fb81211503\") "
Jul 23 00:15:23.747637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.747604    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2btjn\" (UniqueName: \"kubernetes.io/projected/d729d699-83ae-42ee-8c4a-7fafe7e59d90-kube-api-access-2btjn\") pod \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\" (UID: \"d729d699-83ae-42ee-8c4a-7fafe7e59d90\") "
Jul 23 00:15:23.748168 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.748116    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-bundle" (OuterVolumeSpecName: "bundle") pod "ca48f794-daa3-41cf-9f07-56fb81211503" (UID: "ca48f794-daa3-41cf-9f07-56fb81211503"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:23.748241 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.748208    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-bundle" (OuterVolumeSpecName: "bundle") pod "d729d699-83ae-42ee-8c4a-7fafe7e59d90" (UID: "d729d699-83ae-42ee-8c4a-7fafe7e59d90"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:23.750107 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.750079    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca48f794-daa3-41cf-9f07-56fb81211503-kube-api-access-sw6ln" (OuterVolumeSpecName: "kube-api-access-sw6ln") pod "ca48f794-daa3-41cf-9f07-56fb81211503" (UID: "ca48f794-daa3-41cf-9f07-56fb81211503"). InnerVolumeSpecName "kube-api-access-sw6ln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:15:23.750435 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.750410    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d729d699-83ae-42ee-8c4a-7fafe7e59d90-kube-api-access-2btjn" (OuterVolumeSpecName: "kube-api-access-2btjn") pod "d729d699-83ae-42ee-8c4a-7fafe7e59d90" (UID: "d729d699-83ae-42ee-8c4a-7fafe7e59d90"). InnerVolumeSpecName "kube-api-access-2btjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:15:23.755026 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.754951    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-util" (OuterVolumeSpecName: "util") pod "ca48f794-daa3-41cf-9f07-56fb81211503" (UID: "ca48f794-daa3-41cf-9f07-56fb81211503"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:23.755425 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.755394    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-util" (OuterVolumeSpecName: "util") pod "d729d699-83ae-42ee-8c4a-7fafe7e59d90" (UID: "d729d699-83ae-42ee-8c4a-7fafe7e59d90"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:23.848590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.848565    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:23.848590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.848589    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2btjn\" (UniqueName: \"kubernetes.io/projected/d729d699-83ae-42ee-8c4a-7fafe7e59d90-kube-api-access-2btjn\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:23.848717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.848600    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw6ln\" (UniqueName: \"kubernetes.io/projected/ca48f794-daa3-41cf-9f07-56fb81211503-kube-api-access-sw6ln\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:23.848717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.848608    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-util\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:23.848717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.848617    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d729d699-83ae-42ee-8c4a-7fafe7e59d90-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:23.848717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:23.848624    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca48f794-daa3-41cf-9f07-56fb81211503-util\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:24.517096 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.517065    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8"
Jul 23 00:15:24.517507 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.517070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8" event={"ID":"d729d699-83ae-42ee-8c4a-7fafe7e59d90","Type":"ContainerDied","Data":"4e89db3e501d0b7e9226647cc2f79d10671b58d4f86d476913ee7a406aaff379"}
Jul 23 00:15:24.517507 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.517208    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e89db3e501d0b7e9226647cc2f79d10671b58d4f86d476913ee7a406aaff379"
Jul 23 00:15:24.518937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.518906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4" event={"ID":"ca48f794-daa3-41cf-9f07-56fb81211503","Type":"ContainerDied","Data":"0fe758ced0bcb0e001dec43c336b54792a9307a2696607491422c3f839ef51ff"}
Jul 23 00:15:24.519075 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.518947    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0fe758ced0bcb0e001dec43c336b54792a9307a2696607491422c3f839ef51ff"
Jul 23 00:15:24.519075 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.518920    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4"
Jul 23 00:15:24.521060 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.521039    2558 generic.go:356] "Generic (PLEG): container finished" podID="d1a95112-5955-4326-9e24-2ef5e1164095" containerID="f0f034cd835986abd7766d28eb7f130037563f8b86d22e9a99285772ff774f74" exitCode=0
Jul 23 00:15:24.521171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.521107    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw" event={"ID":"d1a95112-5955-4326-9e24-2ef5e1164095","Type":"ContainerDied","Data":"f0f034cd835986abd7766d28eb7f130037563f8b86d22e9a99285772ff774f74"}
Jul 23 00:15:24.649287 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.649266    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:24.756808 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.756783    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-bundle\") pod \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") "
Jul 23 00:15:24.756937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.756821    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntgk5\" (UniqueName: \"kubernetes.io/projected/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-kube-api-access-ntgk5\") pod \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") "
Jul 23 00:15:24.756937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.756867    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-util\") pod \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\" (UID: \"f12feec1-0eb2-4580-9ce4-1f8a0ab96826\") "
Jul 23 00:15:24.757632 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.757603    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-bundle" (OuterVolumeSpecName: "bundle") pod "f12feec1-0eb2-4580-9ce4-1f8a0ab96826" (UID: "f12feec1-0eb2-4580-9ce4-1f8a0ab96826"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:24.758918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.758896    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-kube-api-access-ntgk5" (OuterVolumeSpecName: "kube-api-access-ntgk5") pod "f12feec1-0eb2-4580-9ce4-1f8a0ab96826" (UID: "f12feec1-0eb2-4580-9ce4-1f8a0ab96826"). InnerVolumeSpecName "kube-api-access-ntgk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:15:24.762507 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.762471    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-util" (OuterVolumeSpecName: "util") pod "f12feec1-0eb2-4580-9ce4-1f8a0ab96826" (UID: "f12feec1-0eb2-4580-9ce4-1f8a0ab96826"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:24.858135 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.858113    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:24.858135 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.858133    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntgk5\" (UniqueName: \"kubernetes.io/projected/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-kube-api-access-ntgk5\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:24.858271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:24.858144    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f12feec1-0eb2-4580-9ce4-1f8a0ab96826-util\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:25.528586 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.528550    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh" event={"ID":"f12feec1-0eb2-4580-9ce4-1f8a0ab96826","Type":"ContainerDied","Data":"7b4ae4f866e6318077bf038b4f9b1761bb00963d432de7c4dc492549659cfee9"}
Jul 23 00:15:25.528586 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.528586    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh"
Jul 23 00:15:25.529005 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.528600    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7b4ae4f866e6318077bf038b4f9b1761bb00963d432de7c4dc492549659cfee9"
Jul 23 00:15:25.668288 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.668263    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:25.766515 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.766488    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-util\") pod \"d1a95112-5955-4326-9e24-2ef5e1164095\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") "
Jul 23 00:15:25.766662 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.766539    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-bundle\") pod \"d1a95112-5955-4326-9e24-2ef5e1164095\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") "
Jul 23 00:15:25.766662 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.766584    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2hpk\" (UniqueName: \"kubernetes.io/projected/d1a95112-5955-4326-9e24-2ef5e1164095-kube-api-access-k2hpk\") pod \"d1a95112-5955-4326-9e24-2ef5e1164095\" (UID: \"d1a95112-5955-4326-9e24-2ef5e1164095\") "
Jul 23 00:15:25.767030 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.767008    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-bundle" (OuterVolumeSpecName: "bundle") pod "d1a95112-5955-4326-9e24-2ef5e1164095" (UID: "d1a95112-5955-4326-9e24-2ef5e1164095"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:25.768624 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.768602    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1a95112-5955-4326-9e24-2ef5e1164095-kube-api-access-k2hpk" (OuterVolumeSpecName: "kube-api-access-k2hpk") pod "d1a95112-5955-4326-9e24-2ef5e1164095" (UID: "d1a95112-5955-4326-9e24-2ef5e1164095"). InnerVolumeSpecName "kube-api-access-k2hpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:15:25.772189 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.772151    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-util" (OuterVolumeSpecName: "util") pod "d1a95112-5955-4326-9e24-2ef5e1164095" (UID: "d1a95112-5955-4326-9e24-2ef5e1164095"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:15:25.867582 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.867558    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2hpk\" (UniqueName: \"kubernetes.io/projected/d1a95112-5955-4326-9e24-2ef5e1164095-kube-api-access-k2hpk\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:25.867582 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.867583    2558 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-util\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:25.867694 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:25.867593    2558 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d1a95112-5955-4326-9e24-2ef5e1164095-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:15:26.536875 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:26.536816    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw" event={"ID":"d1a95112-5955-4326-9e24-2ef5e1164095","Type":"ContainerDied","Data":"4f606c06e7361ef35a4728c012bb7b20f47c2f9642c837414cdbc344fe9198e3"}
Jul 23 00:15:26.537254 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:26.536897    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f606c06e7361ef35a4728c012bb7b20f47c2f9642c837414cdbc344fe9198e3"
Jul 23 00:15:26.537254 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:26.536832    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw"
Jul 23 00:15:35.089570 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089534    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"]
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089893    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089904    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089914    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089919    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089927    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089933    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089943    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089948    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089957    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089962    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089968    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089973    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089980    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089985    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089992    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.089998    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090004    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090008    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090017    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090023    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="util"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090029    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090034    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="pull"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090041    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="extract"
Jul 23 00:15:35.090093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090046    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="extract"
Jul 23 00:15:35.090919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090110    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d729d699-83ae-42ee-8c4a-7fafe7e59d90" containerName="extract"
Jul 23 00:15:35.090919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090117    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="d1a95112-5955-4326-9e24-2ef5e1164095" containerName="extract"
Jul 23 00:15:35.090919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090123    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca48f794-daa3-41cf-9f07-56fb81211503" containerName="extract"
Jul 23 00:15:35.090919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.090129    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f12feec1-0eb2-4580-9ce4-1f8a0ab96826" containerName="extract"
Jul 23 00:15:35.094250 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.094233    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:35.096755 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.096735    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 23 00:15:35.096947 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.096927    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c9dpn\""
Jul 23 00:15:35.102757 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.102733    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"]
Jul 23 00:15:35.137989 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.137959    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8nwr\" (UniqueName: \"kubernetes.io/projected/af8d373b-36f5-419a-878a-77d992b312c3-kube-api-access-t8nwr\") pod \"dns-operator-controller-manager-864dd78c4c-mr7kr\" (UID: \"af8d373b-36f5-419a-878a-77d992b312c3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:35.239121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.239093    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8nwr\" (UniqueName: \"kubernetes.io/projected/af8d373b-36f5-419a-878a-77d992b312c3-kube-api-access-t8nwr\") pod \"dns-operator-controller-manager-864dd78c4c-mr7kr\" (UID: \"af8d373b-36f5-419a-878a-77d992b312c3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:35.256041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.256017    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8nwr\" (UniqueName: \"kubernetes.io/projected/af8d373b-36f5-419a-878a-77d992b312c3-kube-api-access-t8nwr\") pod \"dns-operator-controller-manager-864dd78c4c-mr7kr\" (UID: \"af8d373b-36f5-419a-878a-77d992b312c3\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:35.405823 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.405763    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:35.547045 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.547016    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"]
Jul 23 00:15:35.548619 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:35.548593    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf8d373b_36f5_419a_878a_77d992b312c3.slice/crio-3ecc8ef83bc900261e1757aaea9548027ea8c4d6e813d15b4aa06f9d6df092c5 WatchSource:0}: Error finding container 3ecc8ef83bc900261e1757aaea9548027ea8c4d6e813d15b4aa06f9d6df092c5: Status 404 returned error can't find the container with id 3ecc8ef83bc900261e1757aaea9548027ea8c4d6e813d15b4aa06f9d6df092c5
Jul 23 00:15:35.581521 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:35.581494    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr" event={"ID":"af8d373b-36f5-419a-878a-77d992b312c3","Type":"ContainerStarted","Data":"3ecc8ef83bc900261e1757aaea9548027ea8c4d6e813d15b4aa06f9d6df092c5"}
Jul 23 00:15:37.033969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.033930    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5787fc7c8-5xbpf"]
Jul 23 00:15:37.037271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.037250    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.049830 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.049798    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5787fc7c8-5xbpf"]
Jul 23 00:15:37.156437 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156410    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-config\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.156550 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156438    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-trusted-ca-bundle\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.156550 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156461    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-service-ca\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.156550 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156476    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djb54\" (UniqueName: \"kubernetes.io/projected/6ba9964f-ad29-47f3-b715-a5f1adf51653-kube-api-access-djb54\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.156659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156567    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-serving-cert\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.156659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156609    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-oauth-config\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.156659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.156631    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-oauth-serving-cert\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.257909 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.257879    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-serving-cert\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.257920    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-oauth-config\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.257941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-oauth-serving-cert\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258009    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-config\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-trusted-ca-bundle\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258231 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258056    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-service-ca\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258231 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258079    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djb54\" (UniqueName: \"kubernetes.io/projected/6ba9964f-ad29-47f3-b715-a5f1adf51653-kube-api-access-djb54\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258831 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258802    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-oauth-serving-cert\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258818    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-service-ca\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.258982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.258921    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-config\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.259090 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.259006    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6ba9964f-ad29-47f3-b715-a5f1adf51653-trusted-ca-bundle\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.260523 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.260498    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-oauth-config\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.260611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.260530    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6ba9964f-ad29-47f3-b715-a5f1adf51653-console-serving-cert\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.268083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.268063    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djb54\" (UniqueName: \"kubernetes.io/projected/6ba9964f-ad29-47f3-b715-a5f1adf51653-kube-api-access-djb54\") pod \"console-5787fc7c8-5xbpf\" (UID: \"6ba9964f-ad29-47f3-b715-a5f1adf51653\") " pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.346685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.346658    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:37.491383 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:37.491348    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5787fc7c8-5xbpf"]
Jul 23 00:15:37.735886 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:37.735789    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ba9964f_ad29_47f3_b715_a5f1adf51653.slice/crio-e58b233d92452e3b1e6ac365b58d597c2b56c4854f301f24603d341df36e8a71 WatchSource:0}: Error finding container e58b233d92452e3b1e6ac365b58d597c2b56c4854f301f24603d341df36e8a71: Status 404 returned error can't find the container with id e58b233d92452e3b1e6ac365b58d597c2b56c4854f301f24603d341df36e8a71
Jul 23 00:15:38.600480 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:38.600444    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr" event={"ID":"af8d373b-36f5-419a-878a-77d992b312c3","Type":"ContainerStarted","Data":"1171e4984ca5af1d168018ed9216a235f59506438b8f5b83532aafed23934212"}
Jul 23 00:15:38.600866 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:38.600547    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:38.602120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:38.602094    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5787fc7c8-5xbpf" event={"ID":"6ba9964f-ad29-47f3-b715-a5f1adf51653","Type":"ContainerStarted","Data":"d58b8ca8a14f423af666c378159038776218e65e9bb74c8fe52e99d67c6f378b"}
Jul 23 00:15:38.602235 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:38.602131    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5787fc7c8-5xbpf" event={"ID":"6ba9964f-ad29-47f3-b715-a5f1adf51653","Type":"ContainerStarted","Data":"e58b233d92452e3b1e6ac365b58d597c2b56c4854f301f24603d341df36e8a71"}
Jul 23 00:15:38.616548 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:38.616503    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr" podStartSLOduration=1.382190887 podStartE2EDuration="3.616490371s" totalImagesPullingTime="2.234299484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:15:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:15:38.613702068 +0000 UTC m=+688.338017272" watchObservedRunningTime="2026-07-23 00:15:38.616490371 +0000 UTC m=+688.340805555"
Jul 23 00:15:38.631096 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:38.631039    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5787fc7c8-5xbpf" podStartSLOduration=1.631027002 podStartE2EDuration="1.631027002s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:15:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:15:38.629172051 +0000 UTC m=+688.353487275" watchObservedRunningTime="2026-07-23 00:15:38.631027002 +0000 UTC m=+688.355342185"
Jul 23 00:15:40.017564 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.017530    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"]
Jul 23 00:15:40.024555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.024530    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.026859 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.026817    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mdpk6\""
Jul 23 00:15:40.030869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.030847    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"]
Jul 23 00:15:40.082084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.082063    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dwgd\" (UniqueName: \"kubernetes.io/projected/ce5d02b6-09db-407b-a7ee-a1438adcc848-kube-api-access-5dwgd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.082190 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.082092    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce5d02b6-09db-407b-a7ee-a1438adcc848-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.182511 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.182483    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dwgd\" (UniqueName: \"kubernetes.io/projected/ce5d02b6-09db-407b-a7ee-a1438adcc848-kube-api-access-5dwgd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.182618 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.182521    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce5d02b6-09db-407b-a7ee-a1438adcc848-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.182926 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.182907    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce5d02b6-09db-407b-a7ee-a1438adcc848-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.198129 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.198103    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dwgd\" (UniqueName: \"kubernetes.io/projected/ce5d02b6-09db-407b-a7ee-a1438adcc848-kube-api-access-5dwgd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.336903 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.336877    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:40.494770 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.494748    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"]
Jul 23 00:15:40.496893 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:40.496869    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce5d02b6_09db_407b_a7ee_a1438adcc848.slice/crio-3782c1ec8111e3aca6ef797e45f431fcce2fa0329f214ae5164298ce5fe33e14 WatchSource:0}: Error finding container 3782c1ec8111e3aca6ef797e45f431fcce2fa0329f214ae5164298ce5fe33e14: Status 404 returned error can't find the container with id 3782c1ec8111e3aca6ef797e45f431fcce2fa0329f214ae5164298ce5fe33e14
Jul 23 00:15:40.614893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:40.614814    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz" event={"ID":"ce5d02b6-09db-407b-a7ee-a1438adcc848","Type":"ContainerStarted","Data":"3782c1ec8111e3aca6ef797e45f431fcce2fa0329f214ae5164298ce5fe33e14"}
Jul 23 00:15:47.347773 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.347738    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:47.348263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.347807    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:47.353455 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.353431    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:47.655619 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.655516    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz" event={"ID":"ce5d02b6-09db-407b-a7ee-a1438adcc848","Type":"ContainerStarted","Data":"c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570"}
Jul 23 00:15:47.655767 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.655618    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:47.659799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.659778    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5787fc7c8-5xbpf"
Jul 23 00:15:47.671598 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.671555    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz" podStartSLOduration=1.693219765 podStartE2EDuration="8.671542438s" totalImagesPullingTime="6.978322673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:15:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:15:47.67060933 +0000 UTC m=+697.394924514" watchObservedRunningTime="2026-07-23 00:15:47.671542438 +0000 UTC m=+697.395857651"
Jul 23 00:15:47.723072 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:47.723038    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c98d4b46-5llz2"]
Jul 23 00:15:49.610253 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:49.610218    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-mr7kr"
Jul 23 00:15:50.088081 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.088040    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"]
Jul 23 00:15:50.091880 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.091857    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.094497 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.094478    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 23 00:15:50.094989 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.094975    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 23 00:15:50.095057 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.095000    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mrkb7\""
Jul 23 00:15:50.102251 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.102228    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"]
Jul 23 00:15:50.176961 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.176925    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbcpk\" (UniqueName: \"kubernetes.io/projected/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-kube-api-access-rbcpk\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.177139 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.177012    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.177139 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.177034    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.277791 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.277752    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.277791 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.277793    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.278047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.277866    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbcpk\" (UniqueName: \"kubernetes.io/projected/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-kube-api-access-rbcpk\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.278047 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:15:50.277921    2558 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 23 00:15:50.278047 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:15:50.277997    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-plugin-serving-cert podName:2dfc6ebd-a7ef-4969-9a20-42c08a5f0380 nodeName:}" failed. No retries permitted until 2026-07-23 00:15:50.777980441 +0000 UTC m=+700.502295602 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-4gvfd" (UID: "2dfc6ebd-a7ef-4969-9a20-42c08a5f0380") : secret "plugin-serving-cert" not found
Jul 23 00:15:50.278502 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.278483    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.285957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.285934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbcpk\" (UniqueName: \"kubernetes.io/projected/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-kube-api-access-rbcpk\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.783706 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.783669    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:50.786214 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:50.786186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfc6ebd-a7ef-4969-9a20-42c08a5f0380-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-4gvfd\" (UID: \"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:51.001938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:51.001907    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"
Jul 23 00:15:51.131171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:51.131148    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd"]
Jul 23 00:15:51.132429 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:15:51.132405    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dfc6ebd_a7ef_4969_9a20_42c08a5f0380.slice/crio-7f1b7e3b6e7a7ef1025462986ddcc4f098055125614652525ecb8e727d7b3c6e WatchSource:0}: Error finding container 7f1b7e3b6e7a7ef1025462986ddcc4f098055125614652525ecb8e727d7b3c6e: Status 404 returned error can't find the container with id 7f1b7e3b6e7a7ef1025462986ddcc4f098055125614652525ecb8e727d7b3c6e
Jul 23 00:15:51.678048 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:51.678015    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd" event={"ID":"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380","Type":"ContainerStarted","Data":"7f1b7e3b6e7a7ef1025462986ddcc4f098055125614652525ecb8e727d7b3c6e"}
Jul 23 00:15:58.663067 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:58.663036    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:15:59.918110 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:59.918070    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"]
Jul 23 00:15:59.925355 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:59.925332    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:15:59.934923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:15:59.934897    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"]
Jul 23 00:16:00.023058 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.023005    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"]
Jul 23 00:16:00.023358 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:00.023334    2558 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-q5z7f], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg" podUID="bf07d1c4-3000-47a8-bcd6-0a38f1691016"
Jul 23 00:16:00.073226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.073194    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5z7f\" (UniqueName: \"kubernetes.io/projected/bf07d1c4-3000-47a8-bcd6-0a38f1691016-kube-api-access-q5z7f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.073405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.073306    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf07d1c4-3000-47a8-bcd6-0a38f1691016-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.174925 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.174811    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf07d1c4-3000-47a8-bcd6-0a38f1691016-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.175085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.174964    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5z7f\" (UniqueName: \"kubernetes.io/projected/bf07d1c4-3000-47a8-bcd6-0a38f1691016-kube-api-access-q5z7f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.175246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.175221    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf07d1c4-3000-47a8-bcd6-0a38f1691016-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.183081 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.183051    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5z7f\" (UniqueName: \"kubernetes.io/projected/bf07d1c4-3000-47a8-bcd6-0a38f1691016-kube-api-access-q5z7f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.485421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.485344    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"]
Jul 23 00:16:00.498735 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.498705    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"]
Jul 23 00:16:00.513849 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.513816    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"]
Jul 23 00:16:00.514102 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.514052    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" containerName="manager" containerID="cri-o://c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570" gracePeriod=2
Jul 23 00:16:00.529376 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:00.529334    2558 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ce5d02b6-09db-407b-a7ee-a1438adcc848\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-prxrz\": pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:00.532895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.532824    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"]
Jul 23 00:16:00.547596 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.547573    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"]
Jul 23 00:16:00.548159 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.548134    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" containerName="manager"
Jul 23 00:16:00.548159 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.548159    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" containerName="manager"
Jul 23 00:16:00.548322 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.548248    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" containerName="manager"
Jul 23 00:16:00.551873 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.551828    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.560351 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.559913    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"]
Jul 23 00:16:00.571726 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.571703    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"]
Jul 23 00:16:00.576115 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.576025    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:00.577150 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:00.577031    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:00.578337 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.578302    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-9rnlp\""
Jul 23 00:16:00.580446 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:00.580417    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:00.588372 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.588337    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"]
Jul 23 00:16:00.679941 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.679910    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5smmj\" (UniqueName: \"kubernetes.io/projected/a1ad89d8-a8ca-471e-944b-6b502fce77da-kube-api-access-5smmj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zdnhl\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.680674 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.679948    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg7nc\" (UniqueName: \"kubernetes.io/projected/49377f17-0c9d-43ae-85fb-2d85904f8ef6-kube-api-access-wg7nc\") pod \"limitador-operator-controller-manager-859d7b78bd-2rvct\" (UID: \"49377f17-0c9d-43ae-85fb-2d85904f8ef6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:00.680674 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.680116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad89d8-a8ca-471e-944b-6b502fce77da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zdnhl\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.734767 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:00.734728    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:00.738801 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.738741    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.746186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.746166    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.748436 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:00.748408    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="bf07d1c4-3000-47a8-bcd6-0a38f1691016" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:00.780897 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.780867    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5smmj\" (UniqueName: \"kubernetes.io/projected/a1ad89d8-a8ca-471e-944b-6b502fce77da-kube-api-access-5smmj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zdnhl\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.780995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.780917    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg7nc\" (UniqueName: \"kubernetes.io/projected/49377f17-0c9d-43ae-85fb-2d85904f8ef6-kube-api-access-wg7nc\") pod \"limitador-operator-controller-manager-859d7b78bd-2rvct\" (UID: \"49377f17-0c9d-43ae-85fb-2d85904f8ef6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:00.781062 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.781003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad89d8-a8ca-471e-944b-6b502fce77da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zdnhl\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.781422 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.781399    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad89d8-a8ca-471e-944b-6b502fce77da-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zdnhl\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.793233 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.793206    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5smmj\" (UniqueName: \"kubernetes.io/projected/a1ad89d8-a8ca-471e-944b-6b502fce77da-kube-api-access-5smmj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-zdnhl\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.796208 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.796185    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg7nc\" (UniqueName: \"kubernetes.io/projected/49377f17-0c9d-43ae-85fb-2d85904f8ef6-kube-api-access-wg7nc\") pod \"limitador-operator-controller-manager-859d7b78bd-2rvct\" (UID: \"49377f17-0c9d-43ae-85fb-2d85904f8ef6\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:00.881923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.881892    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf07d1c4-3000-47a8-bcd6-0a38f1691016-extensions-socket-volume\") pod \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") "
Jul 23 00:16:00.882065 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.881953    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5z7f\" (UniqueName: \"kubernetes.io/projected/bf07d1c4-3000-47a8-bcd6-0a38f1691016-kube-api-access-q5z7f\") pod \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\" (UID: \"bf07d1c4-3000-47a8-bcd6-0a38f1691016\") "
Jul 23 00:16:00.882162 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.882136    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bf07d1c4-3000-47a8-bcd6-0a38f1691016-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bf07d1c4-3000-47a8-bcd6-0a38f1691016" (UID: "bf07d1c4-3000-47a8-bcd6-0a38f1691016"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:16:00.882407 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.882388    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bf07d1c4-3000-47a8-bcd6-0a38f1691016-extensions-socket-volume\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:00.884226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.884199    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bf07d1c4-3000-47a8-bcd6-0a38f1691016-kube-api-access-q5z7f" (OuterVolumeSpecName: "kube-api-access-q5z7f") pod "bf07d1c4-3000-47a8-bcd6-0a38f1691016" (UID: "bf07d1c4-3000-47a8-bcd6-0a38f1691016"). InnerVolumeSpecName "kube-api-access-q5z7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:16:00.893319 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.893297    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:00.902076 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.902056    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:00.983611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:00.983581    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5z7f\" (UniqueName: \"kubernetes.io/projected/bf07d1c4-3000-47a8-bcd6-0a38f1691016-kube-api-access-q5z7f\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:01.744506 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:01.744474    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:01.746948 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:01.746906    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="bf07d1c4-3000-47a8-bcd6-0a38f1691016" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:01.754031 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:01.754002    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sf2jg\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="bf07d1c4-3000-47a8-bcd6-0a38f1691016" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sf2jg"
Jul 23 00:16:02.736908 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:02.736875    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bf07d1c4-3000-47a8-bcd6-0a38f1691016" path="/var/lib/kubelet/pods/bf07d1c4-3000-47a8-bcd6-0a38f1691016/volumes"
Jul 23 00:16:12.746893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:12.746796    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-77c98d4b46-5llz2" podUID="11470f6a-9f51-4771-b3a2-8cd34b99b91f" containerName="console" containerID="cri-o://0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22" gracePeriod=15
Jul 23 00:16:13.181032 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.180992    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:16:13.184153 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.184134    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:13.220717 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:13.220680    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:13.297465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297406    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-oauth-serving-cert\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.297465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297439    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dwgd\" (UniqueName: \"kubernetes.io/projected/ce5d02b6-09db-407b-a7ee-a1438adcc848-kube-api-access-5dwgd\") pod \"ce5d02b6-09db-407b-a7ee-a1438adcc848\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") "
Jul 23 00:16:13.297599 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297585    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-serving-cert\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.297655 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297631    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-oauth-config\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.297696 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297668    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-trusted-ca-bundle\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.297747 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297703    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:16:13.297747 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297726    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-config\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.297886 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297746    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-service-ca\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.297886 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297790    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce5d02b6-09db-407b-a7ee-a1438adcc848-extensions-socket-volume\") pod \"ce5d02b6-09db-407b-a7ee-a1438adcc848\" (UID: \"ce5d02b6-09db-407b-a7ee-a1438adcc848\") "
Jul 23 00:16:13.297886 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.297867    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqbz7\" (UniqueName: \"kubernetes.io/projected/11470f6a-9f51-4771-b3a2-8cd34b99b91f-kube-api-access-gqbz7\") pod \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\" (UID: \"11470f6a-9f51-4771-b3a2-8cd34b99b91f\") "
Jul 23 00:16:13.298188 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.298134    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-service-ca" (OuterVolumeSpecName: "service-ca") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:16:13.298188 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.298148    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-config" (OuterVolumeSpecName: "console-config") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:16:13.298188 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.298160    2558 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-oauth-serving-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.298469 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.298438    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce5d02b6-09db-407b-a7ee-a1438adcc848-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ce5d02b6-09db-407b-a7ee-a1438adcc848" (UID: "ce5d02b6-09db-407b-a7ee-a1438adcc848"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:16:13.298790 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.298768    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:16:13.299707 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.299687    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:16:13.299769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.299727    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:16:13.299908 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.299890    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce5d02b6-09db-407b-a7ee-a1438adcc848-kube-api-access-5dwgd" (OuterVolumeSpecName: "kube-api-access-5dwgd") pod "ce5d02b6-09db-407b-a7ee-a1438adcc848" (UID: "ce5d02b6-09db-407b-a7ee-a1438adcc848"). InnerVolumeSpecName "kube-api-access-5dwgd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:16:13.300197 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.300178    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11470f6a-9f51-4771-b3a2-8cd34b99b91f-kube-api-access-gqbz7" (OuterVolumeSpecName: "kube-api-access-gqbz7") pod "11470f6a-9f51-4771-b3a2-8cd34b99b91f" (UID: "11470f6a-9f51-4771-b3a2-8cd34b99b91f"). InnerVolumeSpecName "kube-api-access-gqbz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:16:13.399161 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399137    2558 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399161 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399159    2558 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399169    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ce5d02b6-09db-407b-a7ee-a1438adcc848-extensions-socket-volume\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399179    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqbz7\" (UniqueName: \"kubernetes.io/projected/11470f6a-9f51-4771-b3a2-8cd34b99b91f-kube-api-access-gqbz7\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399188    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dwgd\" (UniqueName: \"kubernetes.io/projected/ce5d02b6-09db-407b-a7ee-a1438adcc848-kube-api-access-5dwgd\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399196    2558 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-serving-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399204    2558 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/11470f6a-9f51-4771-b3a2-8cd34b99b91f-console-oauth-config\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.399333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.399212    2558 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/11470f6a-9f51-4771-b3a2-8cd34b99b91f-trusted-ca-bundle\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:13.416608 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.416585    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"]
Jul 23 00:16:13.417399 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:16:13.417371    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49377f17_0c9d_43ae_85fb_2d85904f8ef6.slice/crio-e37e0185fd821007e0447ff315fbf5860f86a0a6799d0bccbe418dec50772c64 WatchSource:0}: Error finding container e37e0185fd821007e0447ff315fbf5860f86a0a6799d0bccbe418dec50772c64: Status 404 returned error can't find the container with id e37e0185fd821007e0447ff315fbf5860f86a0a6799d0bccbe418dec50772c64
Jul 23 00:16:13.421414 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.421395    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"]
Jul 23 00:16:13.422735 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:16:13.422717    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ad89d8_a8ca_471e_944b_6b502fce77da.slice/crio-0057036eb224b3edb46f30cc6d3385875a95e0a522218b1bf2ba9669534f0d31 WatchSource:0}: Error finding container 0057036eb224b3edb46f30cc6d3385875a95e0a522218b1bf2ba9669534f0d31: Status 404 returned error can't find the container with id 0057036eb224b3edb46f30cc6d3385875a95e0a522218b1bf2ba9669534f0d31
Jul 23 00:16:13.821200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.821160    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl" event={"ID":"a1ad89d8-a8ca-471e-944b-6b502fce77da","Type":"ContainerStarted","Data":"80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601"}
Jul 23 00:16:13.821675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.821233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl" event={"ID":"a1ad89d8-a8ca-471e-944b-6b502fce77da","Type":"ContainerStarted","Data":"0057036eb224b3edb46f30cc6d3385875a95e0a522218b1bf2ba9669534f0d31"}
Jul 23 00:16:13.821675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.821318    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:13.822633 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.822595    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct" event={"ID":"49377f17-0c9d-43ae-85fb-2d85904f8ef6","Type":"ContainerStarted","Data":"e37e0185fd821007e0447ff315fbf5860f86a0a6799d0bccbe418dec50772c64"}
Jul 23 00:16:13.824050 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.824029    2558 generic.go:356] "Generic (PLEG): container finished" podID="ce5d02b6-09db-407b-a7ee-a1438adcc848" containerID="c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570" exitCode=0
Jul 23 00:16:13.824144 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.824067    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:13.824144 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.824121    2558 scope.go:117] "RemoveContainer" containerID="c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570"
Jul 23 00:16:13.825417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.825399    2558 generic.go:356] "Generic (PLEG): container finished" podID="11470f6a-9f51-4771-b3a2-8cd34b99b91f" containerID="0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22" exitCode=0
Jul 23 00:16:13.825494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.825428    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c98d4b46-5llz2" event={"ID":"11470f6a-9f51-4771-b3a2-8cd34b99b91f","Type":"ContainerDied","Data":"0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22"}
Jul 23 00:16:13.825494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.825441    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77c98d4b46-5llz2"
Jul 23 00:16:13.825494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.825472    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77c98d4b46-5llz2" event={"ID":"11470f6a-9f51-4771-b3a2-8cd34b99b91f","Type":"ContainerDied","Data":"80061469d305a019ad02bf031aa240d5cd544fc539f4e31654e9aac73a233cd8"}
Jul 23 00:16:13.827180 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.827158    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd" event={"ID":"2dfc6ebd-a7ef-4969-9a20-42c08a5f0380","Type":"ContainerStarted","Data":"7658ceae7017c3620aeda9d3fc3505652d32b72ddff2b6ddb1dd17d865fcf901"}
Jul 23 00:16:13.834331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.834317    2558 scope.go:117] "RemoveContainer" containerID="c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570"
Jul 23 00:16:13.834633 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:13.834607    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570\": container with ID starting with c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570 not found: ID does not exist" containerID="c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570"
Jul 23 00:16:13.834726 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.834643    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570"} err="failed to get container status \"c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570\": rpc error: code = NotFound desc = could not find container \"c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570\": container with ID starting with c817a8eaff3d5f30dd38749e700c13c909a8c8c5acb99852947c0f5a65a04570 not found: ID does not exist"
Jul 23 00:16:13.834726 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.834668    2558 scope.go:117] "RemoveContainer" containerID="0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22"
Jul 23 00:16:13.842299 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:13.842274    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:13.842921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.842884    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl" podStartSLOduration=13.842872141 podStartE2EDuration="13.842872141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:16:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:16:13.840429917 +0000 UTC m=+723.564745101" watchObservedRunningTime="2026-07-23 00:16:13.842872141 +0000 UTC m=+723.567187322"
Jul 23 00:16:13.843257 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.843242    2558 scope.go:117] "RemoveContainer" containerID="0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22"
Jul 23 00:16:13.843511 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:13.843491    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22\": container with ID starting with 0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22 not found: ID does not exist" containerID="0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22"
Jul 23 00:16:13.843575 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.843516    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22"} err="failed to get container status \"0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22\": rpc error: code = NotFound desc = could not find container \"0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22\": container with ID starting with 0f95c8b280ba8636985f23142b56ece64c25eec5c206716a7cecf4d8e90a3a22 not found: ID does not exist"
Jul 23 00:16:13.855997 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.855974    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77c98d4b46-5llz2"]
Jul 23 00:16:13.856751 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:13.856726    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-prxrz\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-prxrz"
Jul 23 00:16:13.858145 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.858127    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77c98d4b46-5llz2"]
Jul 23 00:16:13.871645 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:13.871597    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-4gvfd" podStartSLOduration=1.895374067 podStartE2EDuration="23.871579969s" totalImagesPullingTime="21.976205902s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:15:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:16:13.870532374 +0000 UTC m=+723.594847558" watchObservedRunningTime="2026-07-23 00:16:13.871579969 +0000 UTC m=+723.595895154"
Jul 23 00:16:14.737338 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:14.737300    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11470f6a-9f51-4771-b3a2-8cd34b99b91f" path="/var/lib/kubelet/pods/11470f6a-9f51-4771-b3a2-8cd34b99b91f/volumes"
Jul 23 00:16:14.737851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:14.737820    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce5d02b6-09db-407b-a7ee-a1438adcc848" path="/var/lib/kubelet/pods/ce5d02b6-09db-407b-a7ee-a1438adcc848/volumes"
Jul 23 00:16:15.841721 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:15.841685    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct" event={"ID":"49377f17-0c9d-43ae-85fb-2d85904f8ef6","Type":"ContainerStarted","Data":"a36c596f06b895d69c4112ae31d88caedb41c3751282a943e979665f4c62c393"}
Jul 23 00:16:15.842149 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:15.841773    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:15.858743 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:15.858700    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct" podStartSLOduration=14.038324839 podStartE2EDuration="15.858687856s" totalImagesPullingTime="1.820363017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:16:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:16:15.857730014 +0000 UTC m=+725.582045202" watchObservedRunningTime="2026-07-23 00:16:15.858687856 +0000 UTC m=+725.583003041"
Jul 23 00:16:16.457018 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.456989    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"]
Jul 23 00:16:16.457186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.457164    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl" podUID="a1ad89d8-a8ca-471e-944b-6b502fce77da" containerName="manager" containerID="cri-o://80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601" gracePeriod=10
Jul 23 00:16:16.713337 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.713225    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:16.715826 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.715802    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"]
Jul 23 00:16:16.716266 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.716252    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11470f6a-9f51-4771-b3a2-8cd34b99b91f" containerName="console"
Jul 23 00:16:16.716316 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.716269    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="11470f6a-9f51-4771-b3a2-8cd34b99b91f" containerName="console"
Jul 23 00:16:16.716316 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.716279    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1ad89d8-a8ca-471e-944b-6b502fce77da" containerName="manager"
Jul 23 00:16:16.716316 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.716284    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1ad89d8-a8ca-471e-944b-6b502fce77da" containerName="manager"
Jul 23 00:16:16.716408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.716379    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1ad89d8-a8ca-471e-944b-6b502fce77da" containerName="manager"
Jul 23 00:16:16.716408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.716390    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="11470f6a-9f51-4771-b3a2-8cd34b99b91f" containerName="console"
Jul 23 00:16:16.719684 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.719667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:16.738091 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.738066    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"]
Jul 23 00:16:16.827104 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.827080    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5smmj\" (UniqueName: \"kubernetes.io/projected/a1ad89d8-a8ca-471e-944b-6b502fce77da-kube-api-access-5smmj\") pod \"a1ad89d8-a8ca-471e-944b-6b502fce77da\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") "
Jul 23 00:16:16.827225 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.827185    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad89d8-a8ca-471e-944b-6b502fce77da-extensions-socket-volume\") pod \"a1ad89d8-a8ca-471e-944b-6b502fce77da\" (UID: \"a1ad89d8-a8ca-471e-944b-6b502fce77da\") "
Jul 23 00:16:16.827345 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.827326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b908f31-a478-4354-b83e-d3f0d76533bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m8lmb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:16.827410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.827355    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l2vl\" (UniqueName: \"kubernetes.io/projected/0b908f31-a478-4354-b83e-d3f0d76533bb-kube-api-access-2l2vl\") pod \"kuadrant-operator-controller-manager-86878c769f-m8lmb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:16.827545 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.827522    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1ad89d8-a8ca-471e-944b-6b502fce77da-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a1ad89d8-a8ca-471e-944b-6b502fce77da" (UID: "a1ad89d8-a8ca-471e-944b-6b502fce77da"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:16:16.829069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.829050    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1ad89d8-a8ca-471e-944b-6b502fce77da-kube-api-access-5smmj" (OuterVolumeSpecName: "kube-api-access-5smmj") pod "a1ad89d8-a8ca-471e-944b-6b502fce77da" (UID: "a1ad89d8-a8ca-471e-944b-6b502fce77da"). InnerVolumeSpecName "kube-api-access-5smmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:16:16.848679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.848657    2558 generic.go:356] "Generic (PLEG): container finished" podID="a1ad89d8-a8ca-471e-944b-6b502fce77da" containerID="80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601" exitCode=0
Jul 23 00:16:16.849084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.848698    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"
Jul 23 00:16:16.849084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.848744    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl" event={"ID":"a1ad89d8-a8ca-471e-944b-6b502fce77da","Type":"ContainerDied","Data":"80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601"}
Jul 23 00:16:16.849084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.848801    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl" event={"ID":"a1ad89d8-a8ca-471e-944b-6b502fce77da","Type":"ContainerDied","Data":"0057036eb224b3edb46f30cc6d3385875a95e0a522218b1bf2ba9669534f0d31"}
Jul 23 00:16:16.849084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.848826    2558 scope.go:117] "RemoveContainer" containerID="80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601"
Jul 23 00:16:16.859154 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.859138    2558 scope.go:117] "RemoveContainer" containerID="80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601"
Jul 23 00:16:16.859413 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:16:16.859395    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601\": container with ID starting with 80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601 not found: ID does not exist" containerID="80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601"
Jul 23 00:16:16.859476 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.859421    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601"} err="failed to get container status \"80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601\": rpc error: code = NotFound desc = could not find container \"80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601\": container with ID starting with 80a890cdcbeeda705b07d4804e981004231170dcbf63077bd108ecfccf034601 not found: ID does not exist"
Jul 23 00:16:16.876196 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.876174    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"]
Jul 23 00:16:16.884629 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.884564    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-zdnhl"]
Jul 23 00:16:16.928535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.928513    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b908f31-a478-4354-b83e-d3f0d76533bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m8lmb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:16.928620 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.928543    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l2vl\" (UniqueName: \"kubernetes.io/projected/0b908f31-a478-4354-b83e-d3f0d76533bb-kube-api-access-2l2vl\") pod \"kuadrant-operator-controller-manager-86878c769f-m8lmb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:16.928742 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.928728    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a1ad89d8-a8ca-471e-944b-6b502fce77da-extensions-socket-volume\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:16.928887 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.928869    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5smmj\" (UniqueName: \"kubernetes.io/projected/a1ad89d8-a8ca-471e-944b-6b502fce77da-kube-api-access-5smmj\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:16:16.928961 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.928944    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b908f31-a478-4354-b83e-d3f0d76533bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-m8lmb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:16.939865 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:16.939824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l2vl\" (UniqueName: \"kubernetes.io/projected/0b908f31-a478-4354-b83e-d3f0d76533bb-kube-api-access-2l2vl\") pod \"kuadrant-operator-controller-manager-86878c769f-m8lmb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:17.029274 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:17.029218    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:17.165146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:17.165123    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"]
Jul 23 00:16:17.167404 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:16:17.167380    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b908f31_a478_4354_b83e_d3f0d76533bb.slice/crio-e1872842c062537987bdf552d3578f457fe47468c5bc9edae89843ef31d8958e WatchSource:0}: Error finding container e1872842c062537987bdf552d3578f457fe47468c5bc9edae89843ef31d8958e: Status 404 returned error can't find the container with id e1872842c062537987bdf552d3578f457fe47468c5bc9edae89843ef31d8958e
Jul 23 00:16:17.855964 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:17.855933    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb" event={"ID":"0b908f31-a478-4354-b83e-d3f0d76533bb","Type":"ContainerStarted","Data":"11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a"}
Jul 23 00:16:17.856390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:17.855979    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb" event={"ID":"0b908f31-a478-4354-b83e-d3f0d76533bb","Type":"ContainerStarted","Data":"e1872842c062537987bdf552d3578f457fe47468c5bc9edae89843ef31d8958e"}
Jul 23 00:16:17.856390 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:17.856000    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:17.878287 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:17.878239    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb" podStartSLOduration=1.878223689 podStartE2EDuration="1.878223689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:16:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:16:17.875159384 +0000 UTC m=+727.599474568" watchObservedRunningTime="2026-07-23 00:16:17.878223689 +0000 UTC m=+727.602538873"
Jul 23 00:16:18.735917 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:18.735885    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1ad89d8-a8ca-471e-944b-6b502fce77da" path="/var/lib/kubelet/pods/a1ad89d8-a8ca-471e-944b-6b502fce77da/volumes"
Jul 23 00:16:26.852096 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:26.852055    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2rvct"
Jul 23 00:16:28.864734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:28.864706    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:16:47.945063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:47.945028    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-kbckc"]
Jul 23 00:16:47.978587 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:47.978556    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kbckc"]
Jul 23 00:16:47.978747 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:47.978659    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:16:47.981009 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:47.980987    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-btg5v\""
Jul 23 00:16:48.087520 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:48.087475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7427m\" (UniqueName: \"kubernetes.io/projected/a2592b97-8246-4948-8790-3bb42b4a2272-kube-api-access-7427m\") pod \"authorino-75d556bd77-kbckc\" (UID: \"a2592b97-8246-4948-8790-3bb42b4a2272\") " pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:16:48.188702 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:48.188673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7427m\" (UniqueName: \"kubernetes.io/projected/a2592b97-8246-4948-8790-3bb42b4a2272-kube-api-access-7427m\") pod \"authorino-75d556bd77-kbckc\" (UID: \"a2592b97-8246-4948-8790-3bb42b4a2272\") " pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:16:48.197191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:48.197135    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7427m\" (UniqueName: \"kubernetes.io/projected/a2592b97-8246-4948-8790-3bb42b4a2272-kube-api-access-7427m\") pod \"authorino-75d556bd77-kbckc\" (UID: \"a2592b97-8246-4948-8790-3bb42b4a2272\") " pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:16:48.288568 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:48.288539    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:16:48.416635 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:48.416612    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kbckc"]
Jul 23 00:16:48.418993 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:16:48.418964    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2592b97_8246_4948_8790_3bb42b4a2272.slice/crio-d66f65d9f371179b019df9cb70dbd2e63d61667199968c514c799d4f7575577c WatchSource:0}: Error finding container d66f65d9f371179b019df9cb70dbd2e63d61667199968c514c799d4f7575577c: Status 404 returned error can't find the container with id d66f65d9f371179b019df9cb70dbd2e63d61667199968c514c799d4f7575577c
Jul 23 00:16:49.025506 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:49.025467    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kbckc" event={"ID":"a2592b97-8246-4948-8790-3bb42b4a2272","Type":"ContainerStarted","Data":"d66f65d9f371179b019df9cb70dbd2e63d61667199968c514c799d4f7575577c"}
Jul 23 00:16:52.053018 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:52.052983    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kbckc" event={"ID":"a2592b97-8246-4948-8790-3bb42b4a2272","Type":"ContainerStarted","Data":"505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5"}
Jul 23 00:16:52.066729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:16:52.066682    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-kbckc" podStartSLOduration=2.115297202 podStartE2EDuration="5.066668698s" totalImagesPullingTime="2.951371496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:16:52.066054616 +0000 UTC m=+761.790369821" watchObservedRunningTime="2026-07-23 00:16:52.066668698 +0000 UTC m=+761.790983882"
Jul 23 00:17:21.317330 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.317285    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xpk8m"]
Jul 23 00:17:21.322157 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.322136    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:21.328598 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.328564    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xpk8m"]
Jul 23 00:17:21.364177 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.364149    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzf2s\" (UniqueName: \"kubernetes.io/projected/a8786620-ce33-49da-9c7a-98db5baf5f76-kube-api-access-hzf2s\") pod \"authorino-5d4b67c446-xpk8m\" (UID: \"a8786620-ce33-49da-9c7a-98db5baf5f76\") " pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:21.465384 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.465348    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzf2s\" (UniqueName: \"kubernetes.io/projected/a8786620-ce33-49da-9c7a-98db5baf5f76-kube-api-access-hzf2s\") pod \"authorino-5d4b67c446-xpk8m\" (UID: \"a8786620-ce33-49da-9c7a-98db5baf5f76\") " pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:21.473713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.473694    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzf2s\" (UniqueName: \"kubernetes.io/projected/a8786620-ce33-49da-9c7a-98db5baf5f76-kube-api-access-hzf2s\") pod \"authorino-5d4b67c446-xpk8m\" (UID: \"a8786620-ce33-49da-9c7a-98db5baf5f76\") " pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:21.578161 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.578087    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xpk8m"]
Jul 23 00:17:21.578337 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.578325    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:21.610459 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.610431    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7db854f5d8-d4gcc"]
Jul 23 00:17:21.614032 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.614016    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:21.626377 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.626344    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7db854f5d8-d4gcc"]
Jul 23 00:17:21.666669 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.666632    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8k4z\" (UniqueName: \"kubernetes.io/projected/adf76ff6-7426-4c47-afed-4bfc65b5788f-kube-api-access-m8k4z\") pod \"authorino-7db854f5d8-d4gcc\" (UID: \"adf76ff6-7426-4c47-afed-4bfc65b5788f\") " pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:21.715735 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.715711    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xpk8m"]
Jul 23 00:17:21.716742 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:17:21.716707    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda8786620_ce33_49da_9c7a_98db5baf5f76.slice/crio-3752451148f74948a21e3eca431775263a4db5fd07218f5e07d446e605d1faec WatchSource:0}: Error finding container 3752451148f74948a21e3eca431775263a4db5fd07218f5e07d446e605d1faec: Status 404 returned error can't find the container with id 3752451148f74948a21e3eca431775263a4db5fd07218f5e07d446e605d1faec
Jul 23 00:17:21.767358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.767334    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8k4z\" (UniqueName: \"kubernetes.io/projected/adf76ff6-7426-4c47-afed-4bfc65b5788f-kube-api-access-m8k4z\") pod \"authorino-7db854f5d8-d4gcc\" (UID: \"adf76ff6-7426-4c47-afed-4bfc65b5788f\") " pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:21.776882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.776852    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8k4z\" (UniqueName: \"kubernetes.io/projected/adf76ff6-7426-4c47-afed-4bfc65b5788f-kube-api-access-m8k4z\") pod \"authorino-7db854f5d8-d4gcc\" (UID: \"adf76ff6-7426-4c47-afed-4bfc65b5788f\") " pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:21.924958 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.924885    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:21.953761 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.953726    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db854f5d8-d4gcc"]
Jul 23 00:17:21.993571 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.993545    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-586b4d6f64-pnwfm"]
Jul 23 00:17:21.997290 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:21.997273    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.000148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.000129    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 23 00:17:22.012199 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.012179    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-586b4d6f64-pnwfm"]
Jul 23 00:17:22.068447 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.068421    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db854f5d8-d4gcc"]
Jul 23 00:17:22.069143 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:17:22.069121    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podadf76ff6_7426_4c47_afed_4bfc65b5788f.slice/crio-11791040051c199bdd698e64d1360d44022890349c0b059e3d4a433ad97069f7 WatchSource:0}: Error finding container 11791040051c199bdd698e64d1360d44022890349c0b059e3d4a433ad97069f7: Status 404 returned error can't find the container with id 11791040051c199bdd698e64d1360d44022890349c0b059e3d4a433ad97069f7
Jul 23 00:17:22.070262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.070238    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/39714f1b-7508-46d3-882b-13a0c90fbfae-tls-cert\") pod \"authorino-586b4d6f64-pnwfm\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") " pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.070359 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.070281    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knl9t\" (UniqueName: \"kubernetes.io/projected/39714f1b-7508-46d3-882b-13a0c90fbfae-kube-api-access-knl9t\") pod \"authorino-586b4d6f64-pnwfm\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") " pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.171485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.171455    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/39714f1b-7508-46d3-882b-13a0c90fbfae-tls-cert\") pod \"authorino-586b4d6f64-pnwfm\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") " pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.171637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.171497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knl9t\" (UniqueName: \"kubernetes.io/projected/39714f1b-7508-46d3-882b-13a0c90fbfae-kube-api-access-knl9t\") pod \"authorino-586b4d6f64-pnwfm\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") " pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.173728 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.173705    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/39714f1b-7508-46d3-882b-13a0c90fbfae-tls-cert\") pod \"authorino-586b4d6f64-pnwfm\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") " pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.179992 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.179939    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knl9t\" (UniqueName: \"kubernetes.io/projected/39714f1b-7508-46d3-882b-13a0c90fbfae-kube-api-access-knl9t\") pod \"authorino-586b4d6f64-pnwfm\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") " pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.231059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.231016    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7db854f5d8-d4gcc" event={"ID":"adf76ff6-7426-4c47-afed-4bfc65b5788f","Type":"ContainerStarted","Data":"11791040051c199bdd698e64d1360d44022890349c0b059e3d4a433ad97069f7"}
Jul 23 00:17:22.232497 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.232457    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xpk8m" event={"ID":"a8786620-ce33-49da-9c7a-98db5baf5f76","Type":"ContainerStarted","Data":"f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6"}
Jul 23 00:17:22.232615 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.232508    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xpk8m" event={"ID":"a8786620-ce33-49da-9c7a-98db5baf5f76","Type":"ContainerStarted","Data":"3752451148f74948a21e3eca431775263a4db5fd07218f5e07d446e605d1faec"}
Jul 23 00:17:22.232615 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.232512    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-xpk8m" podUID="a8786620-ce33-49da-9c7a-98db5baf5f76" containerName="authorino" containerID="cri-o://f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6" gracePeriod=30
Jul 23 00:17:22.251255 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.251214    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-xpk8m" podStartSLOduration=0.882076923 podStartE2EDuration="1.251198779s" totalImagesPullingTime="369.121856ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:17:22.248829313 +0000 UTC m=+791.973144516" watchObservedRunningTime="2026-07-23 00:17:22.251198779 +0000 UTC m=+791.975513964"
Jul 23 00:17:22.307162 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.307133    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:17:22.513701 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.513678    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:22.575301 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.575270    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzf2s\" (UniqueName: \"kubernetes.io/projected/a8786620-ce33-49da-9c7a-98db5baf5f76-kube-api-access-hzf2s\") pod \"a8786620-ce33-49da-9c7a-98db5baf5f76\" (UID: \"a8786620-ce33-49da-9c7a-98db5baf5f76\") "
Jul 23 00:17:22.577289 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.577256    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a8786620-ce33-49da-9c7a-98db5baf5f76-kube-api-access-hzf2s" (OuterVolumeSpecName: "kube-api-access-hzf2s") pod "a8786620-ce33-49da-9c7a-98db5baf5f76" (UID: "a8786620-ce33-49da-9c7a-98db5baf5f76"). InnerVolumeSpecName "kube-api-access-hzf2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:17:22.658857 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.658766    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-586b4d6f64-pnwfm"]
Jul 23 00:17:22.661299 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:17:22.661271    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39714f1b_7508_46d3_882b_13a0c90fbfae.slice/crio-3761094c12d3b3b0ef2a80cbe1f4c96b2dcf0a787b4bdacf456c933c32384945 WatchSource:0}: Error finding container 3761094c12d3b3b0ef2a80cbe1f4c96b2dcf0a787b4bdacf456c933c32384945: Status 404 returned error can't find the container with id 3761094c12d3b3b0ef2a80cbe1f4c96b2dcf0a787b4bdacf456c933c32384945
Jul 23 00:17:22.677008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:22.676988    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzf2s\" (UniqueName: \"kubernetes.io/projected/a8786620-ce33-49da-9c7a-98db5baf5f76-kube-api-access-hzf2s\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:17:23.238808 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.238778    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7db854f5d8-d4gcc" event={"ID":"adf76ff6-7426-4c47-afed-4bfc65b5788f","Type":"ContainerStarted","Data":"3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1"}
Jul 23 00:17:23.238953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.238869    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7db854f5d8-d4gcc" podUID="adf76ff6-7426-4c47-afed-4bfc65b5788f" containerName="authorino" containerID="cri-o://3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1" gracePeriod=30
Jul 23 00:17:23.239950 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.239927    2558 generic.go:356] "Generic (PLEG): container finished" podID="a8786620-ce33-49da-9c7a-98db5baf5f76" containerID="f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6" exitCode=0
Jul 23 00:17:23.240063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.239970    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-xpk8m"
Jul 23 00:17:23.240063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.240015    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xpk8m" event={"ID":"a8786620-ce33-49da-9c7a-98db5baf5f76","Type":"ContainerDied","Data":"f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6"}
Jul 23 00:17:23.240063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.240059    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-xpk8m" event={"ID":"a8786620-ce33-49da-9c7a-98db5baf5f76","Type":"ContainerDied","Data":"3752451148f74948a21e3eca431775263a4db5fd07218f5e07d446e605d1faec"}
Jul 23 00:17:23.240245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.240081    2558 scope.go:117] "RemoveContainer" containerID="f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6"
Jul 23 00:17:23.241293 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.241269    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b4d6f64-pnwfm" event={"ID":"39714f1b-7508-46d3-882b-13a0c90fbfae","Type":"ContainerStarted","Data":"3761094c12d3b3b0ef2a80cbe1f4c96b2dcf0a787b4bdacf456c933c32384945"}
Jul 23 00:17:23.248735 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.248720    2558 scope.go:117] "RemoveContainer" containerID="f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6"
Jul 23 00:17:23.249006 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:17:23.248986    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6\": container with ID starting with f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6 not found: ID does not exist" containerID="f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6"
Jul 23 00:17:23.249060 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.249013    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6"} err="failed to get container status \"f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6\": rpc error: code = NotFound desc = could not find container \"f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6\": container with ID starting with f1646d967425d03885cab250aeff8d15ee714ee8491fb1457772e573b7f55ac6 not found: ID does not exist"
Jul 23 00:17:23.256278 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.256240    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7db854f5d8-d4gcc" podStartSLOduration=1.494048673 podStartE2EDuration="2.256228399s" totalImagesPullingTime="762.179726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:17:23.255108761 +0000 UTC m=+792.979423966" watchObservedRunningTime="2026-07-23 00:17:23.256228399 +0000 UTC m=+792.980543583"
Jul 23 00:17:23.271111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.271088    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xpk8m"]
Jul 23 00:17:23.277063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.277045    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-xpk8m"]
Jul 23 00:17:23.544514 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.544485    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:23.686827 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.686799    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8k4z\" (UniqueName: \"kubernetes.io/projected/adf76ff6-7426-4c47-afed-4bfc65b5788f-kube-api-access-m8k4z\") pod \"adf76ff6-7426-4c47-afed-4bfc65b5788f\" (UID: \"adf76ff6-7426-4c47-afed-4bfc65b5788f\") "
Jul 23 00:17:23.688796 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.688775    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/adf76ff6-7426-4c47-afed-4bfc65b5788f-kube-api-access-m8k4z" (OuterVolumeSpecName: "kube-api-access-m8k4z") pod "adf76ff6-7426-4c47-afed-4bfc65b5788f" (UID: "adf76ff6-7426-4c47-afed-4bfc65b5788f"). InnerVolumeSpecName "kube-api-access-m8k4z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:17:23.787541 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:23.787470    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8k4z\" (UniqueName: \"kubernetes.io/projected/adf76ff6-7426-4c47-afed-4bfc65b5788f-kube-api-access-m8k4z\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:17:24.248654 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.248625    2558 generic.go:356] "Generic (PLEG): container finished" podID="adf76ff6-7426-4c47-afed-4bfc65b5788f" containerID="3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1" exitCode=0
Jul 23 00:17:24.248877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.248669    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7db854f5d8-d4gcc"
Jul 23 00:17:24.248877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.248692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7db854f5d8-d4gcc" event={"ID":"adf76ff6-7426-4c47-afed-4bfc65b5788f","Type":"ContainerDied","Data":"3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1"}
Jul 23 00:17:24.248877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.248721    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7db854f5d8-d4gcc" event={"ID":"adf76ff6-7426-4c47-afed-4bfc65b5788f","Type":"ContainerDied","Data":"11791040051c199bdd698e64d1360d44022890349c0b059e3d4a433ad97069f7"}
Jul 23 00:17:24.248877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.248741    2558 scope.go:117] "RemoveContainer" containerID="3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1"
Jul 23 00:17:24.251489 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.251455    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b4d6f64-pnwfm" event={"ID":"39714f1b-7508-46d3-882b-13a0c90fbfae","Type":"ContainerStarted","Data":"260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f"}
Jul 23 00:17:24.258713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.258697    2558 scope.go:117] "RemoveContainer" containerID="3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1"
Jul 23 00:17:24.258980 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:17:24.258959    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1\": container with ID starting with 3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1 not found: ID does not exist" containerID="3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1"
Jul 23 00:17:24.259083 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.258986    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1"} err="failed to get container status \"3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1\": rpc error: code = NotFound desc = could not find container \"3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1\": container with ID starting with 3dd77ecf1ee272a94d5ff4d5f0711d0f412265fdbbd8504756282cd86dd5a0f1 not found: ID does not exist"
Jul 23 00:17:24.266268 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.266146    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-586b4d6f64-pnwfm" podStartSLOduration=2.702491674 podStartE2EDuration="3.266128584s" totalImagesPullingTime="563.63691ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:17:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:17:24.265766079 +0000 UTC m=+793.990081265" watchObservedRunningTime="2026-07-23 00:17:24.266128584 +0000 UTC m=+793.990443769"
Jul 23 00:17:24.282566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.282532    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7db854f5d8-d4gcc"]
Jul 23 00:17:24.284993 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.284971    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7db854f5d8-d4gcc"]
Jul 23 00:17:24.293920 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.293899    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kbckc"]
Jul 23 00:17:24.294073 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.294055    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-kbckc" podUID="a2592b97-8246-4948-8790-3bb42b4a2272" containerName="authorino" containerID="cri-o://505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5" gracePeriod=30
Jul 23 00:17:24.551747 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.551726    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:17:24.695294 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.695268    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7427m\" (UniqueName: \"kubernetes.io/projected/a2592b97-8246-4948-8790-3bb42b4a2272-kube-api-access-7427m\") pod \"a2592b97-8246-4948-8790-3bb42b4a2272\" (UID: \"a2592b97-8246-4948-8790-3bb42b4a2272\") "
Jul 23 00:17:24.697235 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.697212    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2592b97-8246-4948-8790-3bb42b4a2272-kube-api-access-7427m" (OuterVolumeSpecName: "kube-api-access-7427m") pod "a2592b97-8246-4948-8790-3bb42b4a2272" (UID: "a2592b97-8246-4948-8790-3bb42b4a2272"). InnerVolumeSpecName "kube-api-access-7427m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:17:24.736179 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.736155    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a8786620-ce33-49da-9c7a-98db5baf5f76" path="/var/lib/kubelet/pods/a8786620-ce33-49da-9c7a-98db5baf5f76/volumes"
Jul 23 00:17:24.736481 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.736468    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="adf76ff6-7426-4c47-afed-4bfc65b5788f" path="/var/lib/kubelet/pods/adf76ff6-7426-4c47-afed-4bfc65b5788f/volumes"
Jul 23 00:17:24.796520 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:24.796461    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7427m\" (UniqueName: \"kubernetes.io/projected/a2592b97-8246-4948-8790-3bb42b4a2272-kube-api-access-7427m\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:17:25.260436 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.260401    2558 generic.go:356] "Generic (PLEG): container finished" podID="a2592b97-8246-4948-8790-3bb42b4a2272" containerID="505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5" exitCode=0
Jul 23 00:17:25.260630 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.260441    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-kbckc"
Jul 23 00:17:25.260630 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.260470    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kbckc" event={"ID":"a2592b97-8246-4948-8790-3bb42b4a2272","Type":"ContainerDied","Data":"505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5"}
Jul 23 00:17:25.260630 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.260506    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-kbckc" event={"ID":"a2592b97-8246-4948-8790-3bb42b4a2272","Type":"ContainerDied","Data":"d66f65d9f371179b019df9cb70dbd2e63d61667199968c514c799d4f7575577c"}
Jul 23 00:17:25.260630 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.260529    2558 scope.go:117] "RemoveContainer" containerID="505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5"
Jul 23 00:17:25.269849 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.269819    2558 scope.go:117] "RemoveContainer" containerID="505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5"
Jul 23 00:17:25.270083 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:17:25.270064    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5\": container with ID starting with 505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5 not found: ID does not exist" containerID="505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5"
Jul 23 00:17:25.270136 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.270091    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5"} err="failed to get container status \"505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5\": rpc error: code = NotFound desc = could not find container \"505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5\": container with ID starting with 505941b8d70f12955d7cb42b016f0e787605396d6e4ad25b8fc4b9c6c288eab5 not found: ID does not exist"
Jul 23 00:17:25.275999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.275975    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kbckc"]
Jul 23 00:17:25.279328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:25.279307    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-kbckc"]
Jul 23 00:17:26.735360 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:17:26.735324    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2592b97-8246-4948-8790-3bb42b4a2272" path="/var/lib/kubelet/pods/a2592b97-8246-4948-8790-3bb42b4a2272/volumes"
Jul 23 00:18:26.350707 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.350673    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"]
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351069    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2592b97-8246-4948-8790-3bb42b4a2272" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351080    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2592b97-8246-4948-8790-3bb42b4a2272" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351090    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="adf76ff6-7426-4c47-afed-4bfc65b5788f" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351096    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="adf76ff6-7426-4c47-afed-4bfc65b5788f" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351117    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a8786620-ce33-49da-9c7a-98db5baf5f76" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351123    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a8786620-ce33-49da-9c7a-98db5baf5f76" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351191    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a8786620-ce33-49da-9c7a-98db5baf5f76" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351201    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2592b97-8246-4948-8790-3bb42b4a2272" containerName="authorino"
Jul 23 00:18:26.351216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.351217    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="adf76ff6-7426-4c47-afed-4bfc65b5788f" containerName="authorino"
Jul 23 00:18:26.355555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.355533    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.358482 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.358463    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 23 00:18:26.358482 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.358475    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 23 00:18:26.358648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.358477    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-xwdgp\""
Jul 23 00:18:26.358648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.358463    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 23 00:18:26.365673 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.365648    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"]
Jul 23 00:18:26.480171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.480152    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.480282 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.480192    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmnnc\" (UniqueName: \"kubernetes.io/projected/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-kube-api-access-bmnnc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.480333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.480278    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.480333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.480302    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.480412 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.480339    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.480412 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.480359    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581139 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581111    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581148    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581181    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581229    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581434 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581265    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmnnc\" (UniqueName: \"kubernetes.io/projected/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-kube-api-access-bmnnc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581568 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581650 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581566    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.581650 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.581604    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.583495 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.583466    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.583655 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.583636    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.588610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.588576    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmnnc\" (UniqueName: \"kubernetes.io/projected/30fe6e84-f3b6-4556-b9eb-19b15f1e5e43-kube-api-access-bmnnc\") pod \"e2e-distinct-simulated-kserve-59975d69f9-v8m29\" (UID: \"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.667729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.667667    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:26.803309 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:26.803288    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"]
Jul 23 00:18:26.804772 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:18:26.804746    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30fe6e84_f3b6_4556_b9eb_19b15f1e5e43.slice/crio-6884d331fe6c35bc40f9f7401ced0323568b59ee7a4a976f871bfc216eb05753 WatchSource:0}: Error finding container 6884d331fe6c35bc40f9f7401ced0323568b59ee7a4a976f871bfc216eb05753: Status 404 returned error can't find the container with id 6884d331fe6c35bc40f9f7401ced0323568b59ee7a4a976f871bfc216eb05753
Jul 23 00:18:27.643102 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:27.643061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29" event={"ID":"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43","Type":"ContainerStarted","Data":"6884d331fe6c35bc40f9f7401ced0323568b59ee7a4a976f871bfc216eb05753"}
Jul 23 00:18:28.650902 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:28.650862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29" event={"ID":"30fe6e84-f3b6-4556-b9eb-19b15f1e5e43","Type":"ContainerStarted","Data":"205a04a96f9e07f3042ab99320eac2c1c6241ee7454c4d06ce0fff2f17bf0bcb"}
Jul 23 00:18:28.672153 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:28.672100    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29" podStartSLOduration=1.5996420439999999 podStartE2EDuration="2.672085095s" totalImagesPullingTime="1.072443051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:18:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:18:28.670859604 +0000 UTC m=+858.395174812" watchObservedRunningTime="2026-07-23 00:18:28.672085095 +0000 UTC m=+858.396400279"
Jul 23 00:18:36.667802 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:36.667767    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:36.668389 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:36.667830    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:36.680544 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:36.680517    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:36.705502 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:36.705478    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-v8m29"
Jul 23 00:18:51.444124 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.444089    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"]
Jul 23 00:18:51.450659 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.450637    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.453063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.453040    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 23 00:18:51.458146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.458126    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"]
Jul 23 00:18:51.589509 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.589475    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d2llz\" (UniqueName: \"kubernetes.io/projected/d43cfa60-497d-4048-949a-998bd6ee22e0-kube-api-access-d2llz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.589509 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.589524    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.589690 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.589650    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.589690 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.589677    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.589777 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.589710    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cfa60-497d-4048-949a-998bd6ee22e0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.589777 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.589735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.690460 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.690430    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cfa60-497d-4048-949a-998bd6ee22e0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.690590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.690467    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.690590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.690497    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d2llz\" (UniqueName: \"kubernetes.io/projected/d43cfa60-497d-4048-949a-998bd6ee22e0-kube-api-access-d2llz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.690590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.690534    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.690713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.690634    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.690713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.690661    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.691291 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.691219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.692602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.691398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.692602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.691428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.696967 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.693474    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d43cfa60-497d-4048-949a-998bd6ee22e0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.696967 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.693897    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d43cfa60-497d-4048-949a-998bd6ee22e0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.699988 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.699961    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d2llz\" (UniqueName: \"kubernetes.io/projected/d43cfa60-497d-4048-949a-998bd6ee22e0-kube-api-access-d2llz\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-bj8jq\" (UID: \"d43cfa60-497d-4048-949a-998bd6ee22e0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.763613 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.763589    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:18:51.925211 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:51.925187    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"]
Jul 23 00:18:51.927057 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:18:51.927027    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd43cfa60_497d_4048_949a_998bd6ee22e0.slice/crio-052cc5c5a5291b4d383bb0571e6f8c58ad599c2b88d6d98be7e7582de1d0482c WatchSource:0}: Error finding container 052cc5c5a5291b4d383bb0571e6f8c58ad599c2b88d6d98be7e7582de1d0482c: Status 404 returned error can't find the container with id 052cc5c5a5291b4d383bb0571e6f8c58ad599c2b88d6d98be7e7582de1d0482c
Jul 23 00:18:52.788726 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:52.788611    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq" event={"ID":"d43cfa60-497d-4048-949a-998bd6ee22e0","Type":"ContainerStarted","Data":"bcff0fda9f96858f8e6e2e0aead02524be0ff7b11550c4e19e3bb8d6a48dcf01"}
Jul 23 00:18:52.788726 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:52.788670    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq" event={"ID":"d43cfa60-497d-4048-949a-998bd6ee22e0","Type":"ContainerStarted","Data":"052cc5c5a5291b4d383bb0571e6f8c58ad599c2b88d6d98be7e7582de1d0482c"}
Jul 23 00:18:52.807906 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:18:52.807862    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq" podStartSLOduration=1.6419268360000001 podStartE2EDuration="1.807847807s" totalImagesPullingTime="165.920971ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:18:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:18:52.806269858 +0000 UTC m=+882.530585043" watchObservedRunningTime="2026-07-23 00:18:52.807847807 +0000 UTC m=+882.532162981"
Jul 23 00:19:01.763896 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:19:01.763860    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:19:01.764352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:19:01.763914    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:19:01.776720 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:19:01.776698    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:19:01.850319 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:19:01.850299    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-bj8jq"
Jul 23 00:19:10.702119 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:19:10.702042    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:19:10.704242 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:19:10.704218    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:20:01.569552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.569510    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-586b4d6f64-pnwfm"]
Jul 23 00:20:01.572184 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.569724    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-586b4d6f64-pnwfm" podUID="39714f1b-7508-46d3-882b-13a0c90fbfae" containerName="authorino" containerID="cri-o://260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f" gracePeriod=30
Jul 23 00:20:01.822527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.822467    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:20:01.963461 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.963432    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/39714f1b-7508-46d3-882b-13a0c90fbfae-tls-cert\") pod \"39714f1b-7508-46d3-882b-13a0c90fbfae\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") "
Jul 23 00:20:01.963599 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.963517    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knl9t\" (UniqueName: \"kubernetes.io/projected/39714f1b-7508-46d3-882b-13a0c90fbfae-kube-api-access-knl9t\") pod \"39714f1b-7508-46d3-882b-13a0c90fbfae\" (UID: \"39714f1b-7508-46d3-882b-13a0c90fbfae\") "
Jul 23 00:20:01.965349 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.965309    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39714f1b-7508-46d3-882b-13a0c90fbfae-kube-api-access-knl9t" (OuterVolumeSpecName: "kube-api-access-knl9t") pod "39714f1b-7508-46d3-882b-13a0c90fbfae" (UID: "39714f1b-7508-46d3-882b-13a0c90fbfae"). InnerVolumeSpecName "kube-api-access-knl9t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:20:01.975088 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:01.975055    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39714f1b-7508-46d3-882b-13a0c90fbfae-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "39714f1b-7508-46d3-882b-13a0c90fbfae" (UID: "39714f1b-7508-46d3-882b-13a0c90fbfae"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:20:02.064752 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.064724    2558 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/39714f1b-7508-46d3-882b-13a0c90fbfae-tls-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:20:02.064752 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.064749    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knl9t\" (UniqueName: \"kubernetes.io/projected/39714f1b-7508-46d3-882b-13a0c90fbfae-kube-api-access-knl9t\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:20:02.179277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.179205    2558 generic.go:356] "Generic (PLEG): container finished" podID="39714f1b-7508-46d3-882b-13a0c90fbfae" containerID="260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f" exitCode=0
Jul 23 00:20:02.179277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.179265    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-586b4d6f64-pnwfm"
Jul 23 00:20:02.179457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.179262    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b4d6f64-pnwfm" event={"ID":"39714f1b-7508-46d3-882b-13a0c90fbfae","Type":"ContainerDied","Data":"260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f"}
Jul 23 00:20:02.179457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.179331    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-586b4d6f64-pnwfm" event={"ID":"39714f1b-7508-46d3-882b-13a0c90fbfae","Type":"ContainerDied","Data":"3761094c12d3b3b0ef2a80cbe1f4c96b2dcf0a787b4bdacf456c933c32384945"}
Jul 23 00:20:02.179457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.179358    2558 scope.go:117] "RemoveContainer" containerID="260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f"
Jul 23 00:20:02.189191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.189176    2558 scope.go:117] "RemoveContainer" containerID="260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f"
Jul 23 00:20:02.189447 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:20:02.189430    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f\": container with ID starting with 260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f not found: ID does not exist" containerID="260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f"
Jul 23 00:20:02.189492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.189456    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f"} err="failed to get container status \"260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f\": rpc error: code = NotFound desc = could not find container \"260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f\": container with ID starting with 260d001fc255c40c1edfe28319359a008d47739ecc08cec697f03aacb8a79c5f not found: ID does not exist"
Jul 23 00:20:02.201771 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.201749    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-586b4d6f64-pnwfm"]
Jul 23 00:20:02.205566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.205548    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-586b4d6f64-pnwfm"]
Jul 23 00:20:02.735602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:20:02.735572    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39714f1b-7508-46d3-882b-13a0c90fbfae" path="/var/lib/kubelet/pods/39714f1b-7508-46d3-882b-13a0c90fbfae/volumes"
Jul 23 00:24:10.739465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:24:10.739390    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:24:10.742372 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:24:10.742343    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:27:01.632733 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:01.632664    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"]
Jul 23 00:27:01.635315 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:01.632906    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb" podUID="0b908f31-a478-4354-b83e-d3f0d76533bb" containerName="manager" containerID="cri-o://11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a" gracePeriod=10
Jul 23 00:27:05.196609 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.196585    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:27:05.346895 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.346868    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l2vl\" (UniqueName: \"kubernetes.io/projected/0b908f31-a478-4354-b83e-d3f0d76533bb-kube-api-access-2l2vl\") pod \"0b908f31-a478-4354-b83e-d3f0d76533bb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") "
Jul 23 00:27:05.347053 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.346993    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b908f31-a478-4354-b83e-d3f0d76533bb-extensions-socket-volume\") pod \"0b908f31-a478-4354-b83e-d3f0d76533bb\" (UID: \"0b908f31-a478-4354-b83e-d3f0d76533bb\") "
Jul 23 00:27:05.347378 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.347353    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b908f31-a478-4354-b83e-d3f0d76533bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "0b908f31-a478-4354-b83e-d3f0d76533bb" (UID: "0b908f31-a478-4354-b83e-d3f0d76533bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:27:05.348796 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.348776    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b908f31-a478-4354-b83e-d3f0d76533bb-kube-api-access-2l2vl" (OuterVolumeSpecName: "kube-api-access-2l2vl") pod "0b908f31-a478-4354-b83e-d3f0d76533bb" (UID: "0b908f31-a478-4354-b83e-d3f0d76533bb"). InnerVolumeSpecName "kube-api-access-2l2vl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:27:05.448484 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.448463    2558 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/0b908f31-a478-4354-b83e-d3f0d76533bb-extensions-socket-volume\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:27:05.448484 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.448485    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l2vl\" (UniqueName: \"kubernetes.io/projected/0b908f31-a478-4354-b83e-d3f0d76533bb-kube-api-access-2l2vl\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:27:05.518856 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.518808    2558 generic.go:356] "Generic (PLEG): container finished" podID="0b908f31-a478-4354-b83e-d3f0d76533bb" containerID="11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a" exitCode=0
Jul 23 00:27:05.518968 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.518881    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"
Jul 23 00:27:05.518968 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.518892    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb" event={"ID":"0b908f31-a478-4354-b83e-d3f0d76533bb","Type":"ContainerDied","Data":"11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a"}
Jul 23 00:27:05.518968 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.518945    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb" event={"ID":"0b908f31-a478-4354-b83e-d3f0d76533bb","Type":"ContainerDied","Data":"e1872842c062537987bdf552d3578f457fe47468c5bc9edae89843ef31d8958e"}
Jul 23 00:27:05.518968 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.518968    2558 scope.go:117] "RemoveContainer" containerID="11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a"
Jul 23 00:27:05.529291 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.529274    2558 scope.go:117] "RemoveContainer" containerID="11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a"
Jul 23 00:27:05.529528 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:27:05.529510    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a\": container with ID starting with 11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a not found: ID does not exist" containerID="11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a"
Jul 23 00:27:05.529591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.529535    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a"} err="failed to get container status \"11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a\": rpc error: code = NotFound desc = could not find container \"11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a\": container with ID starting with 11737034c444bdb161cd5bbf6b034c4af03e84239de6c16f173380a9326c970a not found: ID does not exist"
Jul 23 00:27:05.540328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.540308    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"]
Jul 23 00:27:05.544826 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:05.544806    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-m8lmb"]
Jul 23 00:27:06.735244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:27:06.735214    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b908f31-a478-4354-b83e-d3f0d76533bb" path="/var/lib/kubelet/pods/0b908f31-a478-4354-b83e-d3f0d76533bb/volumes"
Jul 23 00:28:02.933957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.933924    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"]
Jul 23 00:28:02.934416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.934303    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39714f1b-7508-46d3-882b-13a0c90fbfae" containerName="authorino"
Jul 23 00:28:02.934416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.934314    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="39714f1b-7508-46d3-882b-13a0c90fbfae" containerName="authorino"
Jul 23 00:28:02.934416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.934326    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b908f31-a478-4354-b83e-d3f0d76533bb" containerName="manager"
Jul 23 00:28:02.934416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.934331    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b908f31-a478-4354-b83e-d3f0d76533bb" containerName="manager"
Jul 23 00:28:02.934416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.934393    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b908f31-a478-4354-b83e-d3f0d76533bb" containerName="manager"
Jul 23 00:28:02.934416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.934403    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="39714f1b-7508-46d3-882b-13a0c90fbfae" containerName="authorino"
Jul 23 00:28:02.937460 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.937445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:02.940567 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.940547    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-mdpk6\""
Jul 23 00:28:02.953198 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:02.953171    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"]
Jul 23 00:28:03.028617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.028591    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8p7jt\" (UniqueName: \"kubernetes.io/projected/d3e20902-9c43-4ca3-8c48-79cf3d2d8e71-kube-api-access-8p7jt\") pod \"kuadrant-operator-controller-manager-86878c769f-96nvp\" (UID: \"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.028708 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.028649    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3e20902-9c43-4ca3-8c48-79cf3d2d8e71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-96nvp\" (UID: \"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.129233 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.129173    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8p7jt\" (UniqueName: \"kubernetes.io/projected/d3e20902-9c43-4ca3-8c48-79cf3d2d8e71-kube-api-access-8p7jt\") pod \"kuadrant-operator-controller-manager-86878c769f-96nvp\" (UID: \"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.129233 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.129225    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3e20902-9c43-4ca3-8c48-79cf3d2d8e71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-96nvp\" (UID: \"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.129599 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.129581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d3e20902-9c43-4ca3-8c48-79cf3d2d8e71-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-96nvp\" (UID: \"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.144519 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.144492    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8p7jt\" (UniqueName: \"kubernetes.io/projected/d3e20902-9c43-4ca3-8c48-79cf3d2d8e71-kube-api-access-8p7jt\") pod \"kuadrant-operator-controller-manager-86878c769f-96nvp\" (UID: \"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.247113 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.247061    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.401127 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.401102    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"]
Jul 23 00:28:03.403212 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:28:03.403180    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3e20902_9c43_4ca3_8c48_79cf3d2d8e71.slice/crio-76eb7bd3c67f963dc950b819c4851ba9507141c19e073910f3333443ced78e68 WatchSource:0}: Error finding container 76eb7bd3c67f963dc950b819c4851ba9507141c19e073910f3333443ced78e68: Status 404 returned error can't find the container with id 76eb7bd3c67f963dc950b819c4851ba9507141c19e073910f3333443ced78e68
Jul 23 00:28:03.869982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.869942    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp" event={"ID":"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71","Type":"ContainerStarted","Data":"aec566743d2d3a02d7111cfd9d441fccd26c0c3d92a3abced7472fec4d1650fc"}
Jul 23 00:28:03.870169 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.870014    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp" event={"ID":"d3e20902-9c43-4ca3-8c48-79cf3d2d8e71","Type":"ContainerStarted","Data":"76eb7bd3c67f963dc950b819c4851ba9507141c19e073910f3333443ced78e68"}
Jul 23 00:28:03.870169 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.870067    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:28:03.904691 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:03.904639    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp" podStartSLOduration=1.904626161 podStartE2EDuration="1.904626161s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:28:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:03.902457764 +0000 UTC m=+1433.626772983" watchObservedRunningTime="2026-07-23 00:28:03.904626161 +0000 UTC m=+1433.628941344"
Jul 23 00:28:14.877693 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:28:14.877663    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-96nvp"
Jul 23 00:29:10.774333 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:29:10.774306    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:29:10.778649 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:29:10.778623    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:30:00.139560 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.139489    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"]
Jul 23 00:30:00.143080 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.143064    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"
Jul 23 00:30:00.145347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.145330    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:30:00.152313 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.152296    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"
Jul 23 00:30:00.161566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.161542    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"]
Jul 23 00:30:00.289314 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.289282    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"]
Jul 23 00:30:00.289525 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:30:00.289495    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d234332_9be7_49a5_a2be_3e27459ee4ac.slice/crio-4f1b3bbf89f394b5e1680ed254d8d3e4b56f52778fc501a02f58ed8b6a358c51 WatchSource:0}: Error finding container 4f1b3bbf89f394b5e1680ed254d8d3e4b56f52778fc501a02f58ed8b6a358c51: Status 404 returned error can't find the container with id 4f1b3bbf89f394b5e1680ed254d8d3e4b56f52778fc501a02f58ed8b6a358c51
Jul 23 00:30:00.291563 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.291547    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:30:00.514788 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:00.514723    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr" event={"ID":"5d234332-9be7-49a5-a2be-3e27459ee4ac","Type":"ContainerStarted","Data":"4f1b3bbf89f394b5e1680ed254d8d3e4b56f52778fc501a02f58ed8b6a358c51"}
Jul 23 00:30:02.527991 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:02.527937    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr" event={"ID":"5d234332-9be7-49a5-a2be-3e27459ee4ac","Type":"ContainerStarted","Data":"5166203a93f7316a0cfb8f30bcc76385447019e44e49b4999bdd73baf5fcc2f3"}
Jul 23 00:30:02.543931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:02.543887    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr" podStartSLOduration=0.557947698 podStartE2EDuration="2.543868231s" totalImagesPullingTime="1.985920533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:30:02.543413712 +0000 UTC m=+1552.267728897" watchObservedRunningTime="2026-07-23 00:30:02.543868231 +0000 UTC m=+1552.268183411"
Jul 23 00:30:03.535341 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:03.535264    2558 generic.go:356] "Generic (PLEG): container finished" podID="5d234332-9be7-49a5-a2be-3e27459ee4ac" containerID="5166203a93f7316a0cfb8f30bcc76385447019e44e49b4999bdd73baf5fcc2f3" exitCode=0
Jul 23 00:30:03.535685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:03.535351    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr" event={"ID":"5d234332-9be7-49a5-a2be-3e27459ee4ac","Type":"ContainerDied","Data":"5166203a93f7316a0cfb8f30bcc76385447019e44e49b4999bdd73baf5fcc2f3"}
Jul 23 00:30:04.673204 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:04.673180    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"
Jul 23 00:30:05.549820 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:05.549788    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr"
Jul 23 00:30:05.550093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:05.549822    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29746110-8jcxr" event={"ID":"5d234332-9be7-49a5-a2be-3e27459ee4ac","Type":"ContainerDied","Data":"4f1b3bbf89f394b5e1680ed254d8d3e4b56f52778fc501a02f58ed8b6a358c51"}
Jul 23 00:30:05.550093 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:05.549892    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4f1b3bbf89f394b5e1680ed254d8d3e4b56f52778fc501a02f58ed8b6a358c51"
Jul 23 00:30:40.855374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.855345    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"]
Jul 23 00:30:40.855815 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.855728    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d234332-9be7-49a5-a2be-3e27459ee4ac" containerName="cleanup"
Jul 23 00:30:40.855815 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.855740    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d234332-9be7-49a5-a2be-3e27459ee4ac" containerName="cleanup"
Jul 23 00:30:40.855815 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.855801    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d234332-9be7-49a5-a2be-3e27459ee4ac" containerName="cleanup"
Jul 23 00:30:40.861710 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.861693    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:40.864022 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.864002    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-a5386f29-kserve-self-signed-certs\""
Jul 23 00:30:40.869674 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.869646    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"]
Jul 23 00:30:40.944012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.943970    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-tmp-dir\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:40.944144 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.944016    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh8t5\" (UniqueName: \"kubernetes.io/projected/1206d2c7-c9cb-43af-a0ac-cc7726413472-kube-api-access-lh8t5\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:40.944144 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.944116    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-dshm\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:40.944223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.944162    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-model-cache\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:40.944223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.944180    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1206d2c7-c9cb-43af-a0ac-cc7726413472-tls-certs\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:40.944292 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:40.944231    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-home\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045058 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045027    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-model-cache\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045058 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045061    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1206d2c7-c9cb-43af-a0ac-cc7726413472-tls-certs\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045093    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-home\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045135    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-tmp-dir\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh8t5\" (UniqueName: \"kubernetes.io/projected/1206d2c7-c9cb-43af-a0ac-cc7726413472-kube-api-access-lh8t5\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045243    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-dshm\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045488 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045468    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-model-cache\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045511    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-home\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.045590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.045547    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-tmp-dir\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.047410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.047389    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-dshm\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.047648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.047631    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1206d2c7-c9cb-43af-a0ac-cc7726413472-tls-certs\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.054073 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.054050    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh8t5\" (UniqueName: \"kubernetes.io/projected/1206d2c7-c9cb-43af-a0ac-cc7726413472-kube-api-access-lh8t5\") pod \"e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") " pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.172388 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.172327    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:41.322310 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:30:41.322280    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1206d2c7_c9cb_43af_a0ac_cc7726413472.slice/crio-44ecdb7e348e66f4e31505bc96c8c5215220114ed2c1efdf607acbc467d60ede WatchSource:0}: Error finding container 44ecdb7e348e66f4e31505bc96c8c5215220114ed2c1efdf607acbc467d60ede: Status 404 returned error can't find the container with id 44ecdb7e348e66f4e31505bc96c8c5215220114ed2c1efdf607acbc467d60ede
Jul 23 00:30:41.322658 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.322633    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"]
Jul 23 00:30:41.756331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.756288    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx" event={"ID":"1206d2c7-c9cb-43af-a0ac-cc7726413472","Type":"ContainerStarted","Data":"44a268f8b7b0ef52b4820d18a48b543c9f349d7852608345de689c441d8614ff"}
Jul 23 00:30:41.756331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.756329    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx" event={"ID":"1206d2c7-c9cb-43af-a0ac-cc7726413472","Type":"ContainerStarted","Data":"44ecdb7e348e66f4e31505bc96c8c5215220114ed2c1efdf607acbc467d60ede"}
Jul 23 00:30:41.778799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:41.778751    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx" podStartSLOduration=1.7787370409999999 podStartE2EDuration="1.778737041s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:30:41.777607306 +0000 UTC m=+1591.501922491" watchObservedRunningTime="2026-07-23 00:30:41.778737041 +0000 UTC m=+1591.503052224"
Jul 23 00:30:43.938524 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.938492    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"]
Jul 23 00:30:43.942062 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.942041    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:43.944482 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.944462    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-a5386f29-kserve-self-signed-certs\""
Jul 23 00:30:43.953251 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.953229    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"]
Jul 23 00:30:43.965670 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.965642    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tls-certs\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:43.965768 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.965702    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-model-cache\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:43.965768 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.965739    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwpvw\" (UniqueName: \"kubernetes.io/projected/e57df6fd-ca38-4a31-ad6b-bde883b46edf-kube-api-access-rwpvw\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:43.965862 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.965772    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-home\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:43.965862 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.965820    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-dshm\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:43.965941 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:43.965880    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tmp-dir\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.066722 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.066694    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-home\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.066884 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.066737    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-dshm\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.066884 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.066758    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tmp-dir\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.066884 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.066821    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tls-certs\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.067033 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.066889    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-model-cache\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.067033 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.066912    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwpvw\" (UniqueName: \"kubernetes.io/projected/e57df6fd-ca38-4a31-ad6b-bde883b46edf-kube-api-access-rwpvw\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.067149 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.067110    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-home\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.067207 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.067191    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tmp-dir\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.067264 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.067234    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-model-cache\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.068876 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.068856    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-dshm\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.069176 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.069158    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tls-certs\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.075861 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.075816    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwpvw\" (UniqueName: \"kubernetes.io/projected/e57df6fd-ca38-4a31-ad6b-bde883b46edf-kube-api-access-rwpvw\") pod \"e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") " pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.252510 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.252445    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:44.389298 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.389270    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"]
Jul 23 00:30:44.391431 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:30:44.391405    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode57df6fd_ca38_4a31_ad6b_bde883b46edf.slice/crio-2e1756f0efc3d1f9e91afa0ecf634c3878b4e71bd4f429278a764225b905c58a WatchSource:0}: Error finding container 2e1756f0efc3d1f9e91afa0ecf634c3878b4e71bd4f429278a764225b905c58a: Status 404 returned error can't find the container with id 2e1756f0efc3d1f9e91afa0ecf634c3878b4e71bd4f429278a764225b905c58a
Jul 23 00:30:44.773979 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.773940    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx" event={"ID":"e57df6fd-ca38-4a31-ad6b-bde883b46edf","Type":"ContainerStarted","Data":"b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549"}
Jul 23 00:30:44.774176 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.773991    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx" event={"ID":"e57df6fd-ca38-4a31-ad6b-bde883b46edf","Type":"ContainerStarted","Data":"2e1756f0efc3d1f9e91afa0ecf634c3878b4e71bd4f429278a764225b905c58a"}
Jul 23 00:30:44.798009 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:44.797950    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx" podStartSLOduration=1.797933263 podStartE2EDuration="1.797933263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:30:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:30:44.792020114 +0000 UTC m=+1594.516335322" watchObservedRunningTime="2026-07-23 00:30:44.797933263 +0000 UTC m=+1594.522248449"
Jul 23 00:30:51.173527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:51.173487    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:51.173527 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:51.173539    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:51.186215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:51.186189    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:51.823570 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:51.823542    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:30:54.252642 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:54.252609    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:54.253139 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:54.252697    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:54.265015 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:54.264991    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:30:54.839399 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:30:54.839365    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:31:16.869629 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:16.869599    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"]
Jul 23 00:31:17.765488 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:17.765454    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"]
Jul 23 00:31:31.889619 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:31.889558    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx" podUID="1206d2c7-c9cb-43af-a0ac-cc7726413472" containerName="main" containerID="cri-o://44a268f8b7b0ef52b4820d18a48b543c9f349d7852608345de689c441d8614ff" gracePeriod=45
Jul 23 00:31:32.045553 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.045521    2558 generic.go:356] "Generic (PLEG): container finished" podID="1206d2c7-c9cb-43af-a0ac-cc7726413472" containerID="44a268f8b7b0ef52b4820d18a48b543c9f349d7852608345de689c441d8614ff" exitCode=0
Jul 23 00:31:32.045702 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.045592    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx" event={"ID":"1206d2c7-c9cb-43af-a0ac-cc7726413472","Type":"ContainerDied","Data":"44a268f8b7b0ef52b4820d18a48b543c9f349d7852608345de689c441d8614ff"}
Jul 23 00:31:32.146038 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.145986    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:31:32.187223 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187192    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-tmp-dir\") pod \"1206d2c7-c9cb-43af-a0ac-cc7726413472\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") "
Jul 23 00:31:32.187338 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187255    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-model-cache\") pod \"1206d2c7-c9cb-43af-a0ac-cc7726413472\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") "
Jul 23 00:31:32.187338 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187332    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1206d2c7-c9cb-43af-a0ac-cc7726413472-tls-certs\") pod \"1206d2c7-c9cb-43af-a0ac-cc7726413472\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") "
Jul 23 00:31:32.187427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187366    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-home\") pod \"1206d2c7-c9cb-43af-a0ac-cc7726413472\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") "
Jul 23 00:31:32.187427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187401    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1206d2c7-c9cb-43af-a0ac-cc7726413472" (UID: "1206d2c7-c9cb-43af-a0ac-cc7726413472"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:32.187533 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187415    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-dshm\") pod \"1206d2c7-c9cb-43af-a0ac-cc7726413472\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") "
Jul 23 00:31:32.187533 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187504    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-model-cache" (OuterVolumeSpecName: "model-cache") pod "1206d2c7-c9cb-43af-a0ac-cc7726413472" (UID: "1206d2c7-c9cb-43af-a0ac-cc7726413472"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:32.187644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187538    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh8t5\" (UniqueName: \"kubernetes.io/projected/1206d2c7-c9cb-43af-a0ac-cc7726413472-kube-api-access-lh8t5\") pod \"1206d2c7-c9cb-43af-a0ac-cc7726413472\" (UID: \"1206d2c7-c9cb-43af-a0ac-cc7726413472\") "
Jul 23 00:31:32.187704 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187646    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-home" (OuterVolumeSpecName: "home") pod "1206d2c7-c9cb-43af-a0ac-cc7726413472" (UID: "1206d2c7-c9cb-43af-a0ac-cc7726413472"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:32.187956 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187934    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:32.188071 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187979    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:32.188071 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.187992    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:32.189667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.189642    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1206d2c7-c9cb-43af-a0ac-cc7726413472-kube-api-access-lh8t5" (OuterVolumeSpecName: "kube-api-access-lh8t5") pod "1206d2c7-c9cb-43af-a0ac-cc7726413472" (UID: "1206d2c7-c9cb-43af-a0ac-cc7726413472"). InnerVolumeSpecName "kube-api-access-lh8t5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:31:32.189934 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.189913    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-dshm" (OuterVolumeSpecName: "dshm") pod "1206d2c7-c9cb-43af-a0ac-cc7726413472" (UID: "1206d2c7-c9cb-43af-a0ac-cc7726413472"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:32.189994 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.189911    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1206d2c7-c9cb-43af-a0ac-cc7726413472-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1206d2c7-c9cb-43af-a0ac-cc7726413472" (UID: "1206d2c7-c9cb-43af-a0ac-cc7726413472"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:31:32.288438 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.288413    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1206d2c7-c9cb-43af-a0ac-cc7726413472-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:32.288438 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.288435    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1206d2c7-c9cb-43af-a0ac-cc7726413472-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:32.288618 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.288446    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh8t5\" (UniqueName: \"kubernetes.io/projected/1206d2c7-c9cb-43af-a0ac-cc7726413472-kube-api-access-lh8t5\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:32.785480 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:32.785422    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx" podUID="e57df6fd-ca38-4a31-ad6b-bde883b46edf" containerName="main" containerID="cri-o://b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549" gracePeriod=45
Jul 23 00:31:33.041937 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.041905    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:31:33.052870 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.052822    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"
Jul 23 00:31:33.052987 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.052868    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx" event={"ID":"1206d2c7-c9cb-43af-a0ac-cc7726413472","Type":"ContainerDied","Data":"44ecdb7e348e66f4e31505bc96c8c5215220114ed2c1efdf607acbc467d60ede"}
Jul 23 00:31:33.052987 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.052912    2558 scope.go:117] "RemoveContainer" containerID="44a268f8b7b0ef52b4820d18a48b543c9f349d7852608345de689c441d8614ff"
Jul 23 00:31:33.054263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.054240    2558 generic.go:356] "Generic (PLEG): container finished" podID="e57df6fd-ca38-4a31-ad6b-bde883b46edf" containerID="b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549" exitCode=0
Jul 23 00:31:33.054367 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.054278    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"
Jul 23 00:31:33.054367 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.054292    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx" event={"ID":"e57df6fd-ca38-4a31-ad6b-bde883b46edf","Type":"ContainerDied","Data":"b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549"}
Jul 23 00:31:33.054367 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.054320    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx" event={"ID":"e57df6fd-ca38-4a31-ad6b-bde883b46edf","Type":"ContainerDied","Data":"2e1756f0efc3d1f9e91afa0ecf634c3878b4e71bd4f429278a764225b905c58a"}
Jul 23 00:31:33.061757 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.061735    2558 scope.go:117] "RemoveContainer" containerID="b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549"
Jul 23 00:31:33.070541 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.070523    2558 scope.go:117] "RemoveContainer" containerID="b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549"
Jul 23 00:31:33.070906 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:31:33.070878    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549\": container with ID starting with b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549 not found: ID does not exist" containerID="b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549"
Jul 23 00:31:33.071002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.070915    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549"} err="failed to get container status \"b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549\": rpc error: code = NotFound desc = could not find container \"b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549\": container with ID starting with b3818591bc932641799c02c93a1a93998de6e85c77124a10c0c0661aa1f4c549 not found: ID does not exist"
Jul 23 00:31:33.082062 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.082041    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"]
Jul 23 00:31:33.090324 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.090301    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-a5386f29-kserve-7bfb48fdbc-tcgnx"]
Jul 23 00:31:33.094059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094038    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tls-certs\") pod \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") "
Jul 23 00:31:33.094170 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094113    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-model-cache\") pod \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") "
Jul 23 00:31:33.094170 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094155    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-home\") pod \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") "
Jul 23 00:31:33.094288 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094209    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-dshm\") pod \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") "
Jul 23 00:31:33.094288 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094243    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tmp-dir\") pod \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") "
Jul 23 00:31:33.094406 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094311    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwpvw\" (UniqueName: \"kubernetes.io/projected/e57df6fd-ca38-4a31-ad6b-bde883b46edf-kube-api-access-rwpvw\") pod \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\" (UID: \"e57df6fd-ca38-4a31-ad6b-bde883b46edf\") "
Jul 23 00:31:33.094406 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094390    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-model-cache" (OuterVolumeSpecName: "model-cache") pod "e57df6fd-ca38-4a31-ad6b-bde883b46edf" (UID: "e57df6fd-ca38-4a31-ad6b-bde883b46edf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:33.094528 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094508    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-home" (OuterVolumeSpecName: "home") pod "e57df6fd-ca38-4a31-ad6b-bde883b46edf" (UID: "e57df6fd-ca38-4a31-ad6b-bde883b46edf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:33.094690 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094672    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:33.094782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094695    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:33.094782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.094707    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e57df6fd-ca38-4a31-ad6b-bde883b46edf" (UID: "e57df6fd-ca38-4a31-ad6b-bde883b46edf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:33.096124 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.096101    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e57df6fd-ca38-4a31-ad6b-bde883b46edf" (UID: "e57df6fd-ca38-4a31-ad6b-bde883b46edf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:31:33.096480 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.096449    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e57df6fd-ca38-4a31-ad6b-bde883b46edf-kube-api-access-rwpvw" (OuterVolumeSpecName: "kube-api-access-rwpvw") pod "e57df6fd-ca38-4a31-ad6b-bde883b46edf" (UID: "e57df6fd-ca38-4a31-ad6b-bde883b46edf"). InnerVolumeSpecName "kube-api-access-rwpvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:31:33.096600 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.096578    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-dshm" (OuterVolumeSpecName: "dshm") pod "e57df6fd-ca38-4a31-ad6b-bde883b46edf" (UID: "e57df6fd-ca38-4a31-ad6b-bde883b46edf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:31:33.195197 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.195175    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:33.195197 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.195197    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:33.195360 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.195208    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwpvw\" (UniqueName: \"kubernetes.io/projected/e57df6fd-ca38-4a31-ad6b-bde883b46edf-kube-api-access-rwpvw\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:33.195360 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.195216    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e57df6fd-ca38-4a31-ad6b-bde883b46edf-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:31:33.378226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.378203    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"]
Jul 23 00:31:33.400731 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:33.400705    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-a5386f29-kserve-5cf6d57ff-lcnnx"]
Jul 23 00:31:34.734990 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:34.734956    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1206d2c7-c9cb-43af-a0ac-cc7726413472" path="/var/lib/kubelet/pods/1206d2c7-c9cb-43af-a0ac-cc7726413472/volumes"
Jul 23 00:31:34.735341 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:31:34.735327    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e57df6fd-ca38-4a31-ad6b-bde883b46edf" path="/var/lib/kubelet/pods/e57df6fd-ca38-4a31-ad6b-bde883b46edf/volumes"
Jul 23 00:34:10.809991 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:34:10.809963    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:34:10.815210 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:34:10.815192    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:35:32.832508 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832472    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"]
Jul 23 00:35:32.832949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832852    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e57df6fd-ca38-4a31-ad6b-bde883b46edf" containerName="main"
Jul 23 00:35:32.832949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832866    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e57df6fd-ca38-4a31-ad6b-bde883b46edf" containerName="main"
Jul 23 00:35:32.832949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832879    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1206d2c7-c9cb-43af-a0ac-cc7726413472" containerName="main"
Jul 23 00:35:32.832949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832885    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1206d2c7-c9cb-43af-a0ac-cc7726413472" containerName="main"
Jul 23 00:35:32.833098 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832972    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e57df6fd-ca38-4a31-ad6b-bde883b46edf" containerName="main"
Jul 23 00:35:32.833098 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.832983    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1206d2c7-c9cb-43af-a0ac-cc7726413472" containerName="main"
Jul 23 00:35:32.836102 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.836086    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:32.839974 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.839944    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:35:32.840107 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.840026    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:35:32.856268 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.856241    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v\" (UID: \"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:32.859705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.859680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"]
Jul 23 00:35:32.957271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.957239    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v\" (UID: \"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:32.957820 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:32.957801    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v\" (UID: \"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:33.146912 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:33.146783    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:33.280938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:33.280907    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"]
Jul 23 00:35:33.282102 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:35:33.282076    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fb599d0_70b1_43db_b9bc_cfa9cc1d3cbe.slice/crio-571101245a44eadaa4fffb56b5f88e3fc908feca4f580dba223b0b59820ea729 WatchSource:0}: Error finding container 571101245a44eadaa4fffb56b5f88e3fc908feca4f580dba223b0b59820ea729: Status 404 returned error can't find the container with id 571101245a44eadaa4fffb56b5f88e3fc908feca4f580dba223b0b59820ea729
Jul 23 00:35:33.408601 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:33.408539    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v" event={"ID":"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe","Type":"ContainerStarted","Data":"571101245a44eadaa4fffb56b5f88e3fc908feca4f580dba223b0b59820ea729"}
Jul 23 00:35:34.415399 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:34.415364    2558 generic.go:356] "Generic (PLEG): container finished" podID="4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe" containerID="11564b10b91bc18f071ba4c78ca51df62547a6019faa70f1bd98342ea8c60981" exitCode=0
Jul 23 00:35:34.415740 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:34.415429    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v" event={"ID":"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe","Type":"ContainerDied","Data":"11564b10b91bc18f071ba4c78ca51df62547a6019faa70f1bd98342ea8c60981"}
Jul 23 00:35:35.561903 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:35.561882    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:35.578299 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:35.578271    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe-maas-api-service-ca\") pod \"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe\" (UID: \"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe\") "
Jul 23 00:35:35.578662 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:35.578633    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe" (UID: "4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:35:35.679760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:35.679704    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe-maas-api-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:35:36.428745 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:36.428709    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v" event={"ID":"4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe","Type":"ContainerDied","Data":"571101245a44eadaa4fffb56b5f88e3fc908feca4f580dba223b0b59820ea729"}
Jul 23 00:35:36.428745 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:36.428743    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"
Jul 23 00:35:36.429039 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:36.428760    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="571101245a44eadaa4fffb56b5f88e3fc908feca4f580dba223b0b59820ea729"
Jul 23 00:35:47.887829 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:47.887795    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"]
Jul 23 00:35:47.890080 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:47.890053    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-4bcf43ed-dhj5v"]
Jul 23 00:35:48.735219 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:48.735187    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe" path="/var/lib/kubelet/pods/4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe/volumes"
Jul 23 00:35:51.529559 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.529527    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"]
Jul 23 00:35:51.529991 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.529976    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe" containerName="revoke-keys"
Jul 23 00:35:51.530041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.529994    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe" containerName="revoke-keys"
Jul 23 00:35:51.530086 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.530075    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fb599d0-70b1-43db-b9bc-cfa9cc1d3cbe" containerName="revoke-keys"
Jul 23 00:35:51.533455 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.533436    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.535483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.535464    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-04b6cb45-serving-cert\""
Jul 23 00:35:51.535628 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.535601    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 00:35:51.536305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.536289    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6q2qh\""
Jul 23 00:35:51.536399 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.536292    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:35:51.548685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.548666    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"]
Jul 23 00:35:51.612437 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.612411    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4627979-e0d5-4a92-b9c9-f7f39345839c-maas-api-tls\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.612579 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.612525    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4627979-e0d5-4a92-b9c9-f7f39345839c-openshift-service-ca\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.612627 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.612592    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pzcm\" (UniqueName: \"kubernetes.io/projected/a4627979-e0d5-4a92-b9c9-f7f39345839c-kube-api-access-2pzcm\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.713175 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.713151    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pzcm\" (UniqueName: \"kubernetes.io/projected/a4627979-e0d5-4a92-b9c9-f7f39345839c-kube-api-access-2pzcm\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.713281 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.713191    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4627979-e0d5-4a92-b9c9-f7f39345839c-maas-api-tls\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.713281 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.713249    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4627979-e0d5-4a92-b9c9-f7f39345839c-openshift-service-ca\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.713821 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.713801    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4627979-e0d5-4a92-b9c9-f7f39345839c-openshift-service-ca\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.715736 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.715713    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4627979-e0d5-4a92-b9c9-f7f39345839c-maas-api-tls\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.720908 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.720890    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pzcm\" (UniqueName: \"kubernetes.io/projected/a4627979-e0d5-4a92-b9c9-f7f39345839c-kube-api-access-2pzcm\") pod \"maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:51.846814 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:51.846780    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:52.188011 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:52.187984    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"]
Jul 23 00:35:52.189851 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:35:52.189810    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4627979_e0d5_4a92_b9c9_f7f39345839c.slice/crio-c493422bd929e9f11903f0e64e006233d844bebfcd3ebb2ff1ff7d386d8b46dc WatchSource:0}: Error finding container c493422bd929e9f11903f0e64e006233d844bebfcd3ebb2ff1ff7d386d8b46dc: Status 404 returned error can't find the container with id c493422bd929e9f11903f0e64e006233d844bebfcd3ebb2ff1ff7d386d8b46dc
Jul 23 00:35:52.191268 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:52.191249    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:35:52.523107 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:52.523023    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s" event={"ID":"a4627979-e0d5-4a92-b9c9-f7f39345839c","Type":"ContainerStarted","Data":"c493422bd929e9f11903f0e64e006233d844bebfcd3ebb2ff1ff7d386d8b46dc"}
Jul 23 00:35:57.553262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:57.553164    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s" event={"ID":"a4627979-e0d5-4a92-b9c9-f7f39345839c","Type":"ContainerStarted","Data":"50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958"}
Jul 23 00:35:57.553262 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:57.553228    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:35:57.576014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:35:57.575962    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s" podStartSLOduration=1.545366881 podStartE2EDuration="6.575947418s" totalImagesPullingTime="5.030580537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:35:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:35:57.571997066 +0000 UTC m=+1907.296312250" watchObservedRunningTime="2026-07-23 00:35:57.575947418 +0000 UTC m=+1907.300262602"
Jul 23 00:36:03.564780 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:03.564749    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:36:16.516187 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.516155    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"]
Jul 23 00:36:16.519789 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.519768    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:16.522261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.522236    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:36:16.526129 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.526103    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"]
Jul 23 00:36:16.633352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.633326    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx\" (UID: \"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:16.734498 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.734477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx\" (UID: \"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:16.735114 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.735090    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx\" (UID: \"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:16.832304 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:16.832281    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:17.171078 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:17.171052    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"]
Jul 23 00:36:17.172893 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:36:17.172863    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b54e2b7_f233_4e28_a5d7_26cfa7ff44f1.slice/crio-cb94080da4b3a4c0816d318d847845ccb7b69ab62089f9653e80cb8d0b9a15ea WatchSource:0}: Error finding container cb94080da4b3a4c0816d318d847845ccb7b69ab62089f9653e80cb8d0b9a15ea: Status 404 returned error can't find the container with id cb94080da4b3a4c0816d318d847845ccb7b69ab62089f9653e80cb8d0b9a15ea
Jul 23 00:36:17.669370 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:17.669337    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx" event={"ID":"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1","Type":"ContainerStarted","Data":"4e8165c1d10c067e53e51eb5893ff104237a0ee85d20342a39a34b46762419c7"}
Jul 23 00:36:17.669703 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:17.669394    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx" event={"ID":"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1","Type":"ContainerStarted","Data":"cb94080da4b3a4c0816d318d847845ccb7b69ab62089f9653e80cb8d0b9a15ea"}
Jul 23 00:36:17.690056 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:17.690012    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx" podStartSLOduration=1.689997755 podStartE2EDuration="1.689997755s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:36:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:36:17.688505129 +0000 UTC m=+1927.412820314" watchObservedRunningTime="2026-07-23 00:36:17.689997755 +0000 UTC m=+1927.414312938"
Jul 23 00:36:18.676648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:18.676619    2558 generic.go:356] "Generic (PLEG): container finished" podID="8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1" containerID="4e8165c1d10c067e53e51eb5893ff104237a0ee85d20342a39a34b46762419c7" exitCode=0
Jul 23 00:36:18.677127 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:18.676706    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx" event={"ID":"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1","Type":"ContainerDied","Data":"4e8165c1d10c067e53e51eb5893ff104237a0ee85d20342a39a34b46762419c7"}
Jul 23 00:36:19.818230 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:19.818205    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:19.860870 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:19.860828    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1-maas-api-service-ca\") pod \"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1\" (UID: \"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1\") "
Jul 23 00:36:19.861130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:19.861108    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1" (UID: "8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:36:19.962065 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:19.962012    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1-maas-api-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:36:20.691270 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:20.691241    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"
Jul 23 00:36:20.691270 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:20.691254    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx" event={"ID":"8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1","Type":"ContainerDied","Data":"cb94080da4b3a4c0816d318d847845ccb7b69ab62089f9653e80cb8d0b9a15ea"}
Jul 23 00:36:20.691537 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:20.691302    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cb94080da4b3a4c0816d318d847845ccb7b69ab62089f9653e80cb8d0b9a15ea"
Jul 23 00:36:27.054099 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.054059    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"]
Jul 23 00:36:27.054608 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.054333    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s" podUID="a4627979-e0d5-4a92-b9c9-f7f39345839c" containerName="maas-api" containerID="cri-o://50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958" gracePeriod=30
Jul 23 00:36:27.317217 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.317195    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:36:27.411483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.411452    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4627979-e0d5-4a92-b9c9-f7f39345839c-openshift-service-ca\") pod \"a4627979-e0d5-4a92-b9c9-f7f39345839c\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") "
Jul 23 00:36:27.411483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.411487    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4627979-e0d5-4a92-b9c9-f7f39345839c-maas-api-tls\") pod \"a4627979-e0d5-4a92-b9c9-f7f39345839c\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") "
Jul 23 00:36:27.411663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.411504    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pzcm\" (UniqueName: \"kubernetes.io/projected/a4627979-e0d5-4a92-b9c9-f7f39345839c-kube-api-access-2pzcm\") pod \"a4627979-e0d5-4a92-b9c9-f7f39345839c\" (UID: \"a4627979-e0d5-4a92-b9c9-f7f39345839c\") "
Jul 23 00:36:27.411913 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.411878    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a4627979-e0d5-4a92-b9c9-f7f39345839c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a4627979-e0d5-4a92-b9c9-f7f39345839c" (UID: "a4627979-e0d5-4a92-b9c9-f7f39345839c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:36:27.413699 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.413675    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4627979-e0d5-4a92-b9c9-f7f39345839c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a4627979-e0d5-4a92-b9c9-f7f39345839c" (UID: "a4627979-e0d5-4a92-b9c9-f7f39345839c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:36:27.413757 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.413727    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4627979-e0d5-4a92-b9c9-f7f39345839c-kube-api-access-2pzcm" (OuterVolumeSpecName: "kube-api-access-2pzcm") pod "a4627979-e0d5-4a92-b9c9-f7f39345839c" (UID: "a4627979-e0d5-4a92-b9c9-f7f39345839c"). InnerVolumeSpecName "kube-api-access-2pzcm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:36:27.512267 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.512226    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a4627979-e0d5-4a92-b9c9-f7f39345839c-openshift-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:36:27.512267 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.512260    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a4627979-e0d5-4a92-b9c9-f7f39345839c-maas-api-tls\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:36:27.512451 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.512273    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pzcm\" (UniqueName: \"kubernetes.io/projected/a4627979-e0d5-4a92-b9c9-f7f39345839c-kube-api-access-2pzcm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:36:27.732663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.732588    2558 generic.go:356] "Generic (PLEG): container finished" podID="a4627979-e0d5-4a92-b9c9-f7f39345839c" containerID="50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958" exitCode=0
Jul 23 00:36:27.732663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.732618    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"
Jul 23 00:36:27.732826 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.732663    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s" event={"ID":"a4627979-e0d5-4a92-b9c9-f7f39345839c","Type":"ContainerDied","Data":"50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958"}
Jul 23 00:36:27.732826 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.732708    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s" event={"ID":"a4627979-e0d5-4a92-b9c9-f7f39345839c","Type":"ContainerDied","Data":"c493422bd929e9f11903f0e64e006233d844bebfcd3ebb2ff1ff7d386d8b46dc"}
Jul 23 00:36:27.732826 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.732729    2558 scope.go:117] "RemoveContainer" containerID="50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958"
Jul 23 00:36:27.746883 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.746863    2558 scope.go:117] "RemoveContainer" containerID="50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958"
Jul 23 00:36:27.747151 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:36:27.747129    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958\": container with ID starting with 50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958 not found: ID does not exist" containerID="50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958"
Jul 23 00:36:27.747200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.747159    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958"} err="failed to get container status \"50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958\": rpc error: code = NotFound desc = could not find container \"50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958\": container with ID starting with 50cb9feae03138bae1aec8d4bacb91ed0c100e8877087ff79978e1dd46276958 not found: ID does not exist"
Jul 23 00:36:27.755297 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.755276    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"]
Jul 23 00:36:27.758804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:27.758782    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-04b6cb45-6f8bd969fb-vkp7s"]
Jul 23 00:36:28.737060 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:28.737029    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4627979-e0d5-4a92-b9c9-f7f39345839c" path="/var/lib/kubelet/pods/a4627979-e0d5-4a92-b9c9-f7f39345839c/volumes"
Jul 23 00:36:31.609479 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:31.609442    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"]
Jul 23 00:36:31.611875 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:31.611851    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-04b6cb45-m5xrx"]
Jul 23 00:36:32.735943 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:32.735914    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1" path="/var/lib/kubelet/pods/8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1/volumes"
Jul 23 00:36:34.773801 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.773769    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"]
Jul 23 00:36:34.774917 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.774892    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4627979-e0d5-4a92-b9c9-f7f39345839c" containerName="maas-api"
Jul 23 00:36:34.775111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.775095    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4627979-e0d5-4a92-b9c9-f7f39345839c" containerName="maas-api"
Jul 23 00:36:34.775248 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.775234    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1" containerName="revoke-keys"
Jul 23 00:36:34.775331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.775320    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1" containerName="revoke-keys"
Jul 23 00:36:34.775519 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.775507    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b54e2b7-f233-4e28-a5d7-26cfa7ff44f1" containerName="revoke-keys"
Jul 23 00:36:34.775610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.775600    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4627979-e0d5-4a92-b9c9-f7f39345839c" containerName="maas-api"
Jul 23 00:36:34.784881 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.784821    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.787429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.787408    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:36:34.787607 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.787445    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-55ea1e47-serving-cert\""
Jul 23 00:36:34.787823 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.787506    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 00:36:34.787996 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.787882    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6q2qh\""
Jul 23 00:36:34.788265 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.788243    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"]
Jul 23 00:36:34.870476 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.870453    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fafda64-5245-4107-bbb9-648615bdcfdc-openshift-service-ca\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.870602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.870496    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.870602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.870561    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-57tcr\" (UniqueName: \"kubernetes.io/projected/1fafda64-5245-4107-bbb9-648615bdcfdc-kube-api-access-57tcr\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.971407 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.971384    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-57tcr\" (UniqueName: \"kubernetes.io/projected/1fafda64-5245-4107-bbb9-648615bdcfdc-kube-api-access-57tcr\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.971560 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.971477    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fafda64-5245-4107-bbb9-648615bdcfdc-openshift-service-ca\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.971560 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.971520    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.971694 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:36:34.971636    2558 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-serving-cert: secret "maas-api-e2e-ait-55ea1e47-serving-cert" not found
Jul 23 00:36:34.971756 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:36:34.971724    2558 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls podName:1fafda64-5245-4107-bbb9-648615bdcfdc nodeName:}" failed. No retries permitted until 2026-07-23 00:36:35.471702275 +0000 UTC m=+1945.196017454 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls") pod "maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" (UID: "1fafda64-5245-4107-bbb9-648615bdcfdc") : secret "maas-api-e2e-ait-55ea1e47-serving-cert" not found
Jul 23 00:36:34.972306 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.972281    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fafda64-5245-4107-bbb9-648615bdcfdc-openshift-service-ca\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:34.985555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:34.985527    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-57tcr\" (UniqueName: \"kubernetes.io/projected/1fafda64-5245-4107-bbb9-648615bdcfdc-kube-api-access-57tcr\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:35.476522 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:35.476489    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:35.478736 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:35.478704    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls\") pod \"maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:35.697321 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:35.697290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:35.840711 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:35.840680    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"]
Jul 23 00:36:35.841165 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:36:35.841007    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fafda64_5245_4107_bbb9_648615bdcfdc.slice/crio-b9887a63918c190e3d16cb6eac575795ea9ebbf3a574ba9b2da9979bf20edd09 WatchSource:0}: Error finding container b9887a63918c190e3d16cb6eac575795ea9ebbf3a574ba9b2da9979bf20edd09: Status 404 returned error can't find the container with id b9887a63918c190e3d16cb6eac575795ea9ebbf3a574ba9b2da9979bf20edd09
Jul 23 00:36:36.792983 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:36.792944    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" event={"ID":"1fafda64-5245-4107-bbb9-648615bdcfdc","Type":"ContainerStarted","Data":"da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2"}
Jul 23 00:36:36.793130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:36.792999    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" event={"ID":"1fafda64-5245-4107-bbb9-648615bdcfdc","Type":"ContainerStarted","Data":"b9887a63918c190e3d16cb6eac575795ea9ebbf3a574ba9b2da9979bf20edd09"}
Jul 23 00:36:36.793130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:36.793086    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:36:36.819680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:36.819637    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" podStartSLOduration=2.476982978 podStartE2EDuration="2.819624003s" totalImagesPullingTime="342.641025ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:36:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:36:36.816502561 +0000 UTC m=+1946.540817747" watchObservedRunningTime="2026-07-23 00:36:36.819624003 +0000 UTC m=+1946.543939186"
Jul 23 00:36:42.803811 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:36:42.803782    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:37:00.488587 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.488552    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"]
Jul 23 00:37:00.493704 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.493663    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:00.497193 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.497161    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:37:00.502787 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.502756    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"]
Jul 23 00:37:00.588008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.587980    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32aa2922-7f6c-4afa-9988-51e6d509228e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb\" (UID: \"32aa2922-7f6c-4afa-9988-51e6d509228e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:00.688980 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.688952    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32aa2922-7f6c-4afa-9988-51e6d509228e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb\" (UID: \"32aa2922-7f6c-4afa-9988-51e6d509228e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:00.689543 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.689523    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32aa2922-7f6c-4afa-9988-51e6d509228e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb\" (UID: \"32aa2922-7f6c-4afa-9988-51e6d509228e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:00.809726 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:00.809707    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:01.148900 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:01.148872    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"]
Jul 23 00:37:01.150276 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:37:01.150247    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32aa2922_7f6c_4afa_9988_51e6d509228e.slice/crio-9e292c2c8d54e77185b6e2797b95b41f9446c58c62fc2f2d38e8233aa1c08152 WatchSource:0}: Error finding container 9e292c2c8d54e77185b6e2797b95b41f9446c58c62fc2f2d38e8233aa1c08152: Status 404 returned error can't find the container with id 9e292c2c8d54e77185b6e2797b95b41f9446c58c62fc2f2d38e8233aa1c08152
Jul 23 00:37:01.931894 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:01.931860    2558 generic.go:356] "Generic (PLEG): container finished" podID="32aa2922-7f6c-4afa-9988-51e6d509228e" containerID="acb56857a8c9af97148a9fc310d545d8709f034fab09e4558cd904e7fa29bc0a" exitCode=0
Jul 23 00:37:01.932228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:01.931926    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb" event={"ID":"32aa2922-7f6c-4afa-9988-51e6d509228e","Type":"ContainerDied","Data":"acb56857a8c9af97148a9fc310d545d8709f034fab09e4558cd904e7fa29bc0a"}
Jul 23 00:37:01.932228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:01.931975    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb" event={"ID":"32aa2922-7f6c-4afa-9988-51e6d509228e","Type":"ContainerStarted","Data":"9e292c2c8d54e77185b6e2797b95b41f9446c58c62fc2f2d38e8233aa1c08152"}
Jul 23 00:37:03.075327 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.075298    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:03.207435 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.207337    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32aa2922-7f6c-4afa-9988-51e6d509228e-maas-api-service-ca\") pod \"32aa2922-7f6c-4afa-9988-51e6d509228e\" (UID: \"32aa2922-7f6c-4afa-9988-51e6d509228e\") "
Jul 23 00:37:03.207657 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.207634    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32aa2922-7f6c-4afa-9988-51e6d509228e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "32aa2922-7f6c-4afa-9988-51e6d509228e" (UID: "32aa2922-7f6c-4afa-9988-51e6d509228e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:37:03.308007 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.307973    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/32aa2922-7f6c-4afa-9988-51e6d509228e-maas-api-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:03.945260 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.945228    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"
Jul 23 00:37:03.945465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.945232    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb" event={"ID":"32aa2922-7f6c-4afa-9988-51e6d509228e","Type":"ContainerDied","Data":"9e292c2c8d54e77185b6e2797b95b41f9446c58c62fc2f2d38e8233aa1c08152"}
Jul 23 00:37:03.945465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:03.945355    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e292c2c8d54e77185b6e2797b95b41f9446c58c62fc2f2d38e8233aa1c08152"
Jul 23 00:37:11.147496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.147414    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"]
Jul 23 00:37:11.147969 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.147655    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" podUID="1fafda64-5245-4107-bbb9-648615bdcfdc" containerName="maas-api" containerID="cri-o://da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2" gracePeriod=30
Jul 23 00:37:11.398365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.398300    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:37:11.569554 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.569518    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls\") pod \"1fafda64-5245-4107-bbb9-648615bdcfdc\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") "
Jul 23 00:37:11.569704 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.569574    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fafda64-5245-4107-bbb9-648615bdcfdc-openshift-service-ca\") pod \"1fafda64-5245-4107-bbb9-648615bdcfdc\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") "
Jul 23 00:37:11.569704 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.569649    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-57tcr\" (UniqueName: \"kubernetes.io/projected/1fafda64-5245-4107-bbb9-648615bdcfdc-kube-api-access-57tcr\") pod \"1fafda64-5245-4107-bbb9-648615bdcfdc\" (UID: \"1fafda64-5245-4107-bbb9-648615bdcfdc\") "
Jul 23 00:37:11.570013 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.569980    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fafda64-5245-4107-bbb9-648615bdcfdc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1fafda64-5245-4107-bbb9-648615bdcfdc" (UID: "1fafda64-5245-4107-bbb9-648615bdcfdc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:37:11.571602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.571577    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1fafda64-5245-4107-bbb9-648615bdcfdc" (UID: "1fafda64-5245-4107-bbb9-648615bdcfdc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:37:11.571689 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.571635    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fafda64-5245-4107-bbb9-648615bdcfdc-kube-api-access-57tcr" (OuterVolumeSpecName: "kube-api-access-57tcr") pod "1fafda64-5245-4107-bbb9-648615bdcfdc" (UID: "1fafda64-5245-4107-bbb9-648615bdcfdc"). InnerVolumeSpecName "kube-api-access-57tcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:37:11.670742 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.670675    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1fafda64-5245-4107-bbb9-648615bdcfdc-maas-api-tls\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:11.670742 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.670709    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fafda64-5245-4107-bbb9-648615bdcfdc-openshift-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:11.670742 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.670719    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-57tcr\" (UniqueName: \"kubernetes.io/projected/1fafda64-5245-4107-bbb9-648615bdcfdc-kube-api-access-57tcr\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:11.992065 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.991985    2558 generic.go:356] "Generic (PLEG): container finished" podID="1fafda64-5245-4107-bbb9-648615bdcfdc" containerID="da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2" exitCode=0
Jul 23 00:37:11.992065 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.992038    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"
Jul 23 00:37:11.992249 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.992070    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" event={"ID":"1fafda64-5245-4107-bbb9-648615bdcfdc","Type":"ContainerDied","Data":"da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2"}
Jul 23 00:37:11.992249 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.992128    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j" event={"ID":"1fafda64-5245-4107-bbb9-648615bdcfdc","Type":"ContainerDied","Data":"b9887a63918c190e3d16cb6eac575795ea9ebbf3a574ba9b2da9979bf20edd09"}
Jul 23 00:37:11.992249 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:11.992152    2558 scope.go:117] "RemoveContainer" containerID="da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2"
Jul 23 00:37:12.001815 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:12.001796    2558 scope.go:117] "RemoveContainer" containerID="da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2"
Jul 23 00:37:12.002097 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:37:12.002077    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2\": container with ID starting with da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2 not found: ID does not exist" containerID="da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2"
Jul 23 00:37:12.002148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:12.002105    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2"} err="failed to get container status \"da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2\": rpc error: code = NotFound desc = could not find container \"da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2\": container with ID starting with da18fae374da874dbad2e5b5229172df3b6395426b5e094e13fddedbf63f8ea2 not found: ID does not exist"
Jul 23 00:37:12.014565 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:12.014545    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"]
Jul 23 00:37:12.016818 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:12.016797    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-55ea1e47-66b7d968fb-h5v4j"]
Jul 23 00:37:12.735328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:12.735288    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fafda64-5245-4107-bbb9-648615bdcfdc" path="/var/lib/kubelet/pods/1fafda64-5245-4107-bbb9-648615bdcfdc/volumes"
Jul 23 00:37:15.574008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:15.573971    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"]
Jul 23 00:37:15.578893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:15.578863    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-55ea1e47-r54lb"]
Jul 23 00:37:16.735323 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:16.735293    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32aa2922-7f6c-4afa-9988-51e6d509228e" path="/var/lib/kubelet/pods/32aa2922-7f6c-4afa-9988-51e6d509228e/volumes"
Jul 23 00:37:20.181966 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.181912    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"]
Jul 23 00:37:20.182342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.182328    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32aa2922-7f6c-4afa-9988-51e6d509228e" containerName="revoke-keys"
Jul 23 00:37:20.182342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.182342    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="32aa2922-7f6c-4afa-9988-51e6d509228e" containerName="revoke-keys"
Jul 23 00:37:20.182430 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.182351    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fafda64-5245-4107-bbb9-648615bdcfdc" containerName="maas-api"
Jul 23 00:37:20.182430 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.182367    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fafda64-5245-4107-bbb9-648615bdcfdc" containerName="maas-api"
Jul 23 00:37:20.182494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.182441    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fafda64-5245-4107-bbb9-648615bdcfdc" containerName="maas-api"
Jul 23 00:37:20.182494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.182448    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="32aa2922-7f6c-4afa-9988-51e6d509228e" containerName="revoke-keys"
Jul 23 00:37:20.185685 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.185663    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.188061 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.188022    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6q2qh\""
Jul 23 00:37:20.188176 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.188114    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-41658efc-serving-cert\""
Jul 23 00:37:20.188877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.188863    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 23 00:37:20.188940 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.188868    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:37:20.226577 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.226555    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"]
Jul 23 00:37:20.347244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.347209    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a40648-598c-4359-9d00-99754008bbae-maas-api-tls\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.347424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.347290    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a40648-598c-4359-9d00-99754008bbae-openshift-service-ca\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.347424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.347325    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf8bc\" (UniqueName: \"kubernetes.io/projected/f0a40648-598c-4359-9d00-99754008bbae-kube-api-access-jf8bc\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.448362 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.448236    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a40648-598c-4359-9d00-99754008bbae-maas-api-tls\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.448485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.448396    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a40648-598c-4359-9d00-99754008bbae-openshift-service-ca\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.448485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.448430    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf8bc\" (UniqueName: \"kubernetes.io/projected/f0a40648-598c-4359-9d00-99754008bbae-kube-api-access-jf8bc\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.449092 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.449065    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a40648-598c-4359-9d00-99754008bbae-openshift-service-ca\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.450710 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.450685    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a40648-598c-4359-9d00-99754008bbae-maas-api-tls\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.463576 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.463548    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf8bc\" (UniqueName: \"kubernetes.io/projected/f0a40648-598c-4359-9d00-99754008bbae-kube-api-access-jf8bc\") pod \"maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.497205 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.497178    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:20.653667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:20.653643    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"]
Jul 23 00:37:20.655799 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:37:20.655774    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0a40648_598c_4359_9d00_99754008bbae.slice/crio-a803805be1f120bfe347194dcce461ce7cfe8eb6c2aed330d1754f3c0a047817 WatchSource:0}: Error finding container a803805be1f120bfe347194dcce461ce7cfe8eb6c2aed330d1754f3c0a047817: Status 404 returned error can't find the container with id a803805be1f120bfe347194dcce461ce7cfe8eb6c2aed330d1754f3c0a047817
Jul 23 00:37:21.045759 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:21.045722    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg" event={"ID":"f0a40648-598c-4359-9d00-99754008bbae","Type":"ContainerStarted","Data":"a803805be1f120bfe347194dcce461ce7cfe8eb6c2aed330d1754f3c0a047817"}
Jul 23 00:37:22.056450 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:22.056412    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg" event={"ID":"f0a40648-598c-4359-9d00-99754008bbae","Type":"ContainerStarted","Data":"8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b"}
Jul 23 00:37:22.056896 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:22.056513    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:22.096457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:22.096408    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg" podStartSLOduration=1.691244891 podStartE2EDuration="2.096390743s" totalImagesPullingTime="405.145852ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:37:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:37:22.095614707 +0000 UTC m=+1991.819929909" watchObservedRunningTime="2026-07-23 00:37:22.096390743 +0000 UTC m=+1991.820705933"
Jul 23 00:37:28.067307 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:28.067279    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:44.850003 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:44.849968    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"]
Jul 23 00:37:44.853494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:44.853477    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:44.856012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:44.855985    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:37:44.867930 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:44.867908    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"]
Jul 23 00:37:44.939036 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:44.939010    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89adb618-3def-47c9-a2cb-6bde779e9bc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-41658efc-6x7rd\" (UID: \"89adb618-3def-47c9-a2cb-6bde779e9bc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:45.039783 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:45.039762    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89adb618-3def-47c9-a2cb-6bde779e9bc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-41658efc-6x7rd\" (UID: \"89adb618-3def-47c9-a2cb-6bde779e9bc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:45.040391 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:45.040371    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89adb618-3def-47c9-a2cb-6bde779e9bc0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-41658efc-6x7rd\" (UID: \"89adb618-3def-47c9-a2cb-6bde779e9bc0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:45.164011 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:45.163954    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:45.306501 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:45.306477    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"]
Jul 23 00:37:45.308329 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:37:45.308292    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89adb618_3def_47c9_a2cb_6bde779e9bc0.slice/crio-6fb54d5d34ee01023e47e68742f5f0ac2b963b013479d0cc32aadb2a6eff6df3 WatchSource:0}: Error finding container 6fb54d5d34ee01023e47e68742f5f0ac2b963b013479d0cc32aadb2a6eff6df3: Status 404 returned error can't find the container with id 6fb54d5d34ee01023e47e68742f5f0ac2b963b013479d0cc32aadb2a6eff6df3
Jul 23 00:37:46.196877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:46.196823    2558 generic.go:356] "Generic (PLEG): container finished" podID="89adb618-3def-47c9-a2cb-6bde779e9bc0" containerID="208479972a37e8a00e13e9b775eaf5c1ccc7fefb81a4ec18d0877dd373bfc813" exitCode=0
Jul 23 00:37:46.197187 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:46.196908    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd" event={"ID":"89adb618-3def-47c9-a2cb-6bde779e9bc0","Type":"ContainerDied","Data":"208479972a37e8a00e13e9b775eaf5c1ccc7fefb81a4ec18d0877dd373bfc813"}
Jul 23 00:37:46.197187 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:46.196962    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd" event={"ID":"89adb618-3def-47c9-a2cb-6bde779e9bc0","Type":"ContainerStarted","Data":"6fb54d5d34ee01023e47e68742f5f0ac2b963b013479d0cc32aadb2a6eff6df3"}
Jul 23 00:37:47.338555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:47.338531    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:47.461682 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:47.461608    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89adb618-3def-47c9-a2cb-6bde779e9bc0-maas-api-service-ca\") pod \"89adb618-3def-47c9-a2cb-6bde779e9bc0\" (UID: \"89adb618-3def-47c9-a2cb-6bde779e9bc0\") "
Jul 23 00:37:47.461974 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:47.461952    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89adb618-3def-47c9-a2cb-6bde779e9bc0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "89adb618-3def-47c9-a2cb-6bde779e9bc0" (UID: "89adb618-3def-47c9-a2cb-6bde779e9bc0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:37:47.562823 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:47.562799    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/89adb618-3def-47c9-a2cb-6bde779e9bc0-maas-api-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:48.210288 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:48.210261    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"
Jul 23 00:37:48.210463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:48.210301    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd" event={"ID":"89adb618-3def-47c9-a2cb-6bde779e9bc0","Type":"ContainerDied","Data":"6fb54d5d34ee01023e47e68742f5f0ac2b963b013479d0cc32aadb2a6eff6df3"}
Jul 23 00:37:48.210463 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:48.210347    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fb54d5d34ee01023e47e68742f5f0ac2b963b013479d0cc32aadb2a6eff6df3"
Jul 23 00:37:55.551690 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.551656    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"]
Jul 23 00:37:55.552226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.551955    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg" podUID="f0a40648-598c-4359-9d00-99754008bbae" containerName="maas-api" containerID="cri-o://8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b" gracePeriod=30
Jul 23 00:37:55.799962 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.799937    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:55.929756 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.929724    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a40648-598c-4359-9d00-99754008bbae-maas-api-tls\") pod \"f0a40648-598c-4359-9d00-99754008bbae\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") "
Jul 23 00:37:55.929941 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.929772    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a40648-598c-4359-9d00-99754008bbae-openshift-service-ca\") pod \"f0a40648-598c-4359-9d00-99754008bbae\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") "
Jul 23 00:37:55.929941 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.929806    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jf8bc\" (UniqueName: \"kubernetes.io/projected/f0a40648-598c-4359-9d00-99754008bbae-kube-api-access-jf8bc\") pod \"f0a40648-598c-4359-9d00-99754008bbae\" (UID: \"f0a40648-598c-4359-9d00-99754008bbae\") "
Jul 23 00:37:55.930115 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.930094    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0a40648-598c-4359-9d00-99754008bbae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0a40648-598c-4359-9d00-99754008bbae" (UID: "f0a40648-598c-4359-9d00-99754008bbae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:37:55.930200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.930185    2558 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0a40648-598c-4359-9d00-99754008bbae-openshift-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:55.931930 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.931906    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0a40648-598c-4359-9d00-99754008bbae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0a40648-598c-4359-9d00-99754008bbae" (UID: "f0a40648-598c-4359-9d00-99754008bbae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:37:55.931982 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:55.931954    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0a40648-598c-4359-9d00-99754008bbae-kube-api-access-jf8bc" (OuterVolumeSpecName: "kube-api-access-jf8bc") pod "f0a40648-598c-4359-9d00-99754008bbae" (UID: "f0a40648-598c-4359-9d00-99754008bbae"). InnerVolumeSpecName "kube-api-access-jf8bc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:37:56.031455 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.031428    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0a40648-598c-4359-9d00-99754008bbae-maas-api-tls\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:56.031455 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.031451    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jf8bc\" (UniqueName: \"kubernetes.io/projected/f0a40648-598c-4359-9d00-99754008bbae-kube-api-access-jf8bc\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:37:56.267146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.267066    2558 generic.go:356] "Generic (PLEG): container finished" podID="f0a40648-598c-4359-9d00-99754008bbae" containerID="8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b" exitCode=0
Jul 23 00:37:56.267146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.267123    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"
Jul 23 00:37:56.267319 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.267152    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg" event={"ID":"f0a40648-598c-4359-9d00-99754008bbae","Type":"ContainerDied","Data":"8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b"}
Jul 23 00:37:56.267319 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.267226    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg" event={"ID":"f0a40648-598c-4359-9d00-99754008bbae","Type":"ContainerDied","Data":"a803805be1f120bfe347194dcce461ce7cfe8eb6c2aed330d1754f3c0a047817"}
Jul 23 00:37:56.267319 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.267249    2558 scope.go:117] "RemoveContainer" containerID="8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b"
Jul 23 00:37:56.278398 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.278380    2558 scope.go:117] "RemoveContainer" containerID="8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b"
Jul 23 00:37:56.278690 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:37:56.278671    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b\": container with ID starting with 8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b not found: ID does not exist" containerID="8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b"
Jul 23 00:37:56.278737 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.278698    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b"} err="failed to get container status \"8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b\": rpc error: code = NotFound desc = could not find container \"8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b\": container with ID starting with 8f3beb5ef0c2fdb64b34c050f7abaa40bdc6774f3e52e1c1e3fc1e66cd1dc87b not found: ID does not exist"
Jul 23 00:37:56.292060 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.292038    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"]
Jul 23 00:37:56.294284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.294264    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-41658efc-d974b6cf4-5cfzg"]
Jul 23 00:37:56.735572 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:56.735544    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0a40648-598c-4359-9d00-99754008bbae" path="/var/lib/kubelet/pods/f0a40648-598c-4359-9d00-99754008bbae/volumes"
Jul 23 00:37:59.918923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:59.918874    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"]
Jul 23 00:37:59.921198 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:37:59.921176    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-41658efc-6x7rd"]
Jul 23 00:38:00.735261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:38:00.735235    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89adb618-3def-47c9-a2cb-6bde779e9bc0" path="/var/lib/kubelet/pods/89adb618-3def-47c9-a2cb-6bde779e9bc0/volumes"
Jul 23 00:39:10.848370 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:39:10.848293    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:39:10.854806 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:39:10.854788    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:40:00.175922 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.175888    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"]
Jul 23 00:40:00.176347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.176279    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89adb618-3def-47c9-a2cb-6bde779e9bc0" containerName="revoke-keys"
Jul 23 00:40:00.176347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.176291    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="89adb618-3def-47c9-a2cb-6bde779e9bc0" containerName="revoke-keys"
Jul 23 00:40:00.176347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.176313    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0a40648-598c-4359-9d00-99754008bbae" containerName="maas-api"
Jul 23 00:40:00.176347 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.176321    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0a40648-598c-4359-9d00-99754008bbae" containerName="maas-api"
Jul 23 00:40:00.176492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.176394    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="89adb618-3def-47c9-a2cb-6bde779e9bc0" containerName="revoke-keys"
Jul 23 00:40:00.176492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.176404    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0a40648-598c-4359-9d00-99754008bbae" containerName="maas-api"
Jul 23 00:40:00.179747 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.179727    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:00.182116 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.182088    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:40:00.182229 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.182188    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:40:00.187957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.187932    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"]
Jul 23 00:40:00.244175 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.244148    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f471fcb-39c6-4bc8-a636-101ca3926e3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp\" (UID: \"3f471fcb-39c6-4bc8-a636-101ca3926e3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:00.345467 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.345436    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f471fcb-39c6-4bc8-a636-101ca3926e3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp\" (UID: \"3f471fcb-39c6-4bc8-a636-101ca3926e3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:00.346055 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.346036    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f471fcb-39c6-4bc8-a636-101ca3926e3b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp\" (UID: \"3f471fcb-39c6-4bc8-a636-101ca3926e3b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:00.491910 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.491826    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:00.836406 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.836383    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"]
Jul 23 00:40:00.838157 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:40:00.838130    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f471fcb_39c6_4bc8_a636_101ca3926e3b.slice/crio-eb16cc216f446292f72702ca98526f9e99605a4850c021563e315a2e7f63595a WatchSource:0}: Error finding container eb16cc216f446292f72702ca98526f9e99605a4850c021563e315a2e7f63595a: Status 404 returned error can't find the container with id eb16cc216f446292f72702ca98526f9e99605a4850c021563e315a2e7f63595a
Jul 23 00:40:00.998949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:00.998916    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp" event={"ID":"3f471fcb-39c6-4bc8-a636-101ca3926e3b","Type":"ContainerStarted","Data":"eb16cc216f446292f72702ca98526f9e99605a4850c021563e315a2e7f63595a"}
Jul 23 00:40:02.006068 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:02.005982    2558 generic.go:356] "Generic (PLEG): container finished" podID="3f471fcb-39c6-4bc8-a636-101ca3926e3b" containerID="256aa0f6efcd23f66b56cd9a79694db9133593c0bbca1db9d7125fcee3940021" exitCode=0
Jul 23 00:40:02.006418 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:02.006072    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp" event={"ID":"3f471fcb-39c6-4bc8-a636-101ca3926e3b","Type":"ContainerDied","Data":"256aa0f6efcd23f66b56cd9a79694db9133593c0bbca1db9d7125fcee3940021"}
Jul 23 00:40:03.145505 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:03.145483    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:03.270990 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:03.270929    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f471fcb-39c6-4bc8-a636-101ca3926e3b-maas-api-service-ca\") pod \"3f471fcb-39c6-4bc8-a636-101ca3926e3b\" (UID: \"3f471fcb-39c6-4bc8-a636-101ca3926e3b\") "
Jul 23 00:40:03.271328 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:03.271302    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f471fcb-39c6-4bc8-a636-101ca3926e3b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3f471fcb-39c6-4bc8-a636-101ca3926e3b" (UID: "3f471fcb-39c6-4bc8-a636-101ca3926e3b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:40:03.372020 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:03.371990    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3f471fcb-39c6-4bc8-a636-101ca3926e3b-maas-api-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:40:04.018111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:04.018081    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"
Jul 23 00:40:04.018278 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:04.018111    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp" event={"ID":"3f471fcb-39c6-4bc8-a636-101ca3926e3b","Type":"ContainerDied","Data":"eb16cc216f446292f72702ca98526f9e99605a4850c021563e315a2e7f63595a"}
Jul 23 00:40:04.018278 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:04.018162    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb16cc216f446292f72702ca98526f9e99605a4850c021563e315a2e7f63595a"
Jul 23 00:40:15.364087 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:15.364049    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"]
Jul 23 00:40:15.368113 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:15.368088    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-dfb539bb-nhwnp"]
Jul 23 00:40:16.735961 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:40:16.735931    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f471fcb-39c6-4bc8-a636-101ca3926e3b" path="/var/lib/kubelet/pods/3f471fcb-39c6-4bc8-a636-101ca3926e3b/volumes"
Jul 23 00:41:08.655408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.655375    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"]
Jul 23 00:41:08.655921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.655800    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f471fcb-39c6-4bc8-a636-101ca3926e3b" containerName="revoke-keys"
Jul 23 00:41:08.655921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.655812    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f471fcb-39c6-4bc8-a636-101ca3926e3b" containerName="revoke-keys"
Jul 23 00:41:08.655921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.655901    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f471fcb-39c6-4bc8-a636-101ca3926e3b" containerName="revoke-keys"
Jul 23 00:41:08.659024 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.659007    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.662295 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.662262    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-244e255b\"/\"e2e-lifecycle-model-244e255b-kserve-self-signed-certs\""
Jul 23 00:41:08.662295 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.662291    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-244e255b\"/\"kube-root-ca.crt\""
Jul 23 00:41:08.662492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.662306    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-244e255b\"/\"default-dockercfg-tldqr\""
Jul 23 00:41:08.662492 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.662277    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-244e255b\"/\"openshift-service-ca.crt\""
Jul 23 00:41:08.670402 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.670377    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"]
Jul 23 00:41:08.716359 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.716335    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7dv4\" (UniqueName: \"kubernetes.io/projected/5e546fcd-2183-4c5f-b192-bd74e6f8542e-kube-api-access-z7dv4\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.716464 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.716376    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-model-cache\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.716464 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.716409    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tmp-dir\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.716464 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.716432    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-dshm\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.716573 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.716470    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tls-certs\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.716573 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.716554    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-home\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.817578 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.817550    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-home\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.817679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.817596    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7dv4\" (UniqueName: \"kubernetes.io/projected/5e546fcd-2183-4c5f-b192-bd74e6f8542e-kube-api-access-z7dv4\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.817679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.817643    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-model-cache\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.817679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.817673    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tmp-dir\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.817860 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.817815    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-dshm\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.817944 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.817925    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tls-certs\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.818062 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.818043    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-home\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.818130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.818071    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-model-cache\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.818130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.818099    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tmp-dir\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.820054 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.820032    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-dshm\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.820234 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.820219    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tls-certs\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.826491 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.826469    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7dv4\" (UniqueName: \"kubernetes.io/projected/5e546fcd-2183-4c5f-b192-bd74e6f8542e-kube-api-access-z7dv4\") pod \"e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") " pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:08.970041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:08.969965    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:09.108120 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:09.108097    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"]
Jul 23 00:41:09.110231 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:41:09.110196    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5e546fcd_2183_4c5f_b192_bd74e6f8542e.slice/crio-6f716748d57d34610ab218d47cc8c9120284743fc04b9524a6ff4a6a6053b1fb WatchSource:0}: Error finding container 6f716748d57d34610ab218d47cc8c9120284743fc04b9524a6ff4a6a6053b1fb: Status 404 returned error can't find the container with id 6f716748d57d34610ab218d47cc8c9120284743fc04b9524a6ff4a6a6053b1fb
Jul 23 00:41:09.407420 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:09.407392    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8" event={"ID":"5e546fcd-2183-4c5f-b192-bd74e6f8542e","Type":"ContainerStarted","Data":"5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070"}
Jul 23 00:41:09.407578 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:09.407430    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8" event={"ID":"5e546fcd-2183-4c5f-b192-bd74e6f8542e","Type":"ContainerStarted","Data":"6f716748d57d34610ab218d47cc8c9120284743fc04b9524a6ff4a6a6053b1fb"}
Jul 23 00:41:09.426608 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:09.426562    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8" podStartSLOduration=1.426548656 podStartE2EDuration="1.426548656s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:41:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:41:09.424158102 +0000 UTC m=+2219.148473317" watchObservedRunningTime="2026-07-23 00:41:09.426548656 +0000 UTC m=+2219.150863882"
Jul 23 00:41:18.970710 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:18.970674    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:18.971189 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:18.970723    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:18.983912 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:18.983878    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:41:19.478849 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:41:19.478804    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:42:01.275954 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:01.275915    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"]
Jul 23 00:42:10.954904 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:10.954789    2558 scope.go:117] "RemoveContainer" containerID="11564b10b91bc18f071ba4c78ca51df62547a6019faa70f1bd98342ea8c60981"
Jul 23 00:42:16.296355 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.296304    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8" podUID="5e546fcd-2183-4c5f-b192-bd74e6f8542e" containerName="main" containerID="cri-o://5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070" gracePeriod=45
Jul 23 00:42:16.551021 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.550957    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:42:16.687035 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.686999    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-home\") pod \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") "
Jul 23 00:42:16.687220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687066    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-model-cache\") pod \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") "
Jul 23 00:42:16.687220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687091    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tmp-dir\") pod \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") "
Jul 23 00:42:16.687220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687156    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7dv4\" (UniqueName: \"kubernetes.io/projected/5e546fcd-2183-4c5f-b192-bd74e6f8542e-kube-api-access-z7dv4\") pod \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") "
Jul 23 00:42:16.687220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687188    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-dshm\") pod \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") "
Jul 23 00:42:16.687452 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687265    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tls-certs\") pod \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\" (UID: \"5e546fcd-2183-4c5f-b192-bd74e6f8542e\") "
Jul 23 00:42:16.687452 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687280    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-home" (OuterVolumeSpecName: "home") pod "5e546fcd-2183-4c5f-b192-bd74e6f8542e" (UID: "5e546fcd-2183-4c5f-b192-bd74e6f8542e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:42:16.687452 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687375    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-model-cache" (OuterVolumeSpecName: "model-cache") pod "5e546fcd-2183-4c5f-b192-bd74e6f8542e" (UID: "5e546fcd-2183-4c5f-b192-bd74e6f8542e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:42:16.687452 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687410    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5e546fcd-2183-4c5f-b192-bd74e6f8542e" (UID: "5e546fcd-2183-4c5f-b192-bd74e6f8542e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:42:16.687680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687583    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:42:16.687680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687605    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:42:16.687680 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.687620    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:42:16.689385 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.689360    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-dshm" (OuterVolumeSpecName: "dshm") pod "5e546fcd-2183-4c5f-b192-bd74e6f8542e" (UID: "5e546fcd-2183-4c5f-b192-bd74e6f8542e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:42:16.689461 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.689416    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5e546fcd-2183-4c5f-b192-bd74e6f8542e" (UID: "5e546fcd-2183-4c5f-b192-bd74e6f8542e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:42:16.689514 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.689496    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e546fcd-2183-4c5f-b192-bd74e6f8542e-kube-api-access-z7dv4" (OuterVolumeSpecName: "kube-api-access-z7dv4") pod "5e546fcd-2183-4c5f-b192-bd74e6f8542e" (UID: "5e546fcd-2183-4c5f-b192-bd74e6f8542e"). InnerVolumeSpecName "kube-api-access-z7dv4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:42:16.788631 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.788607    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5e546fcd-2183-4c5f-b192-bd74e6f8542e-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:42:16.788631 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.788626    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7dv4\" (UniqueName: \"kubernetes.io/projected/5e546fcd-2183-4c5f-b192-bd74e6f8542e-kube-api-access-z7dv4\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:42:16.788763 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.788635    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5e546fcd-2183-4c5f-b192-bd74e6f8542e-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:42:16.827986 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.827959    2558 generic.go:356] "Generic (PLEG): container finished" podID="5e546fcd-2183-4c5f-b192-bd74e6f8542e" containerID="5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070" exitCode=0
Jul 23 00:42:16.828085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.827998    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"
Jul 23 00:42:16.828085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.828026    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8" event={"ID":"5e546fcd-2183-4c5f-b192-bd74e6f8542e","Type":"ContainerDied","Data":"5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070"}
Jul 23 00:42:16.828085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.828061    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8" event={"ID":"5e546fcd-2183-4c5f-b192-bd74e6f8542e","Type":"ContainerDied","Data":"6f716748d57d34610ab218d47cc8c9120284743fc04b9524a6ff4a6a6053b1fb"}
Jul 23 00:42:16.828204 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.828087    2558 scope.go:117] "RemoveContainer" containerID="5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070"
Jul 23 00:42:16.837224 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.837210    2558 scope.go:117] "RemoveContainer" containerID="5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070"
Jul 23 00:42:16.837498 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:42:16.837479    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070\": container with ID starting with 5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070 not found: ID does not exist" containerID="5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070"
Jul 23 00:42:16.837563 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.837504    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070"} err="failed to get container status \"5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070\": rpc error: code = NotFound desc = could not find container \"5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070\": container with ID starting with 5bbfb0f0dba6501c1790ca6147e337cd8fba48f729feb61d608ec84643447070 not found: ID does not exist"
Jul 23 00:42:16.848074 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.848046    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"]
Jul 23 00:42:16.852852 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:16.852817    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-244e255b/e2e-lifecycle-model-244e255b-kserve-7b6c84b987-d2qd8"]
Jul 23 00:42:18.735198 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:42:18.735167    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e546fcd-2183-4c5f-b192-bd74e6f8542e" path="/var/lib/kubelet/pods/5e546fcd-2183-4c5f-b192-bd74e6f8542e/volumes"
Jul 23 00:43:10.976780 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:10.976706    2558 scope.go:117] "RemoveContainer" containerID="4e8165c1d10c067e53e51eb5893ff104237a0ee85d20342a39a34b46762419c7"
Jul 23 00:43:10.985605 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:10.985579    2558 scope.go:117] "RemoveContainer" containerID="acb56857a8c9af97148a9fc310d545d8709f034fab09e4558cd904e7fa29bc0a"
Jul 23 00:43:51.808848 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.808811    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"]
Jul 23 00:43:51.809339 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.809245    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e546fcd-2183-4c5f-b192-bd74e6f8542e" containerName="main"
Jul 23 00:43:51.809339 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.809258    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e546fcd-2183-4c5f-b192-bd74e6f8542e" containerName="main"
Jul 23 00:43:51.809423 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.809341    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e546fcd-2183-4c5f-b192-bd74e6f8542e" containerName="main"
Jul 23 00:43:51.812742 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.812721    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.819086 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.819054    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2f7578-openshift-default-dockercfg-w5ptk\""
Jul 23 00:43:51.834748 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.834725    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"]
Jul 23 00:43:51.866701 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866673    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-podinfo\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.866798 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866707    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-data\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.866798 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866735    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-envoy\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.866905 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866806    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1dbf888-3814-49cb-8baf-c856b9282bff-istiod-ca-cert\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.866905 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866875    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-token\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.867002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866908    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-credential-socket\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.867002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866928    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw9j4\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-kube-api-access-jw9j4\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.867002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866949    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-certs\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.867002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.866992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-socket\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967466 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967435    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-credential-socket\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967569 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967471    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw9j4\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-kube-api-access-jw9j4\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967569 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967494    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-certs\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967643 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967605    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-socket\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967761 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967726    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-podinfo\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967830 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967766    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-data\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967830 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967820    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-envoy\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967886    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1dbf888-3814-49cb-8baf-c856b9282bff-istiod-ca-cert\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.967957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967912    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-token\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.968133 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.967970    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-credential-socket\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.968133 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.968025    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-certs\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.968244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.968186    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-socket\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.968369 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.968336    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-data\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.968637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.968616    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1dbf888-3814-49cb-8baf-c856b9282bff-istiod-ca-cert\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.970657 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.970626    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-envoy\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.971056 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.971035    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-podinfo\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.976311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.976290    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-token\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:51.978599 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:51.978581    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw9j4\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-kube-api-access-jw9j4\") pod \"e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") " pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:52.124224 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.124148    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:52.263480 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.263451    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"]
Jul 23 00:43:52.265266 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:43:52.265244    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1dbf888_3814_49cb_8baf_c856b9282bff.slice/crio-20529eed338e98157a305e6150333ef72f1821fab0a697c29e0791cecc1d57ec WatchSource:0}: Error finding container 20529eed338e98157a305e6150333ef72f1821fab0a697c29e0791cecc1d57ec: Status 404 returned error can't find the container with id 20529eed338e98157a305e6150333ef72f1821fab0a697c29e0791cecc1d57ec
Jul 23 00:43:52.267874 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.267831    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:43:52.267954 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.267907    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:43:52.267954 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.267937    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:43:52.372675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.372644    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" event={"ID":"e1dbf888-3814-49cb-8baf-c856b9282bff","Type":"ContainerStarted","Data":"23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"}
Jul 23 00:43:52.372821 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.372706    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" event={"ID":"e1dbf888-3814-49cb-8baf-c856b9282bff","Type":"ContainerStarted","Data":"20529eed338e98157a305e6150333ef72f1821fab0a697c29e0791cecc1d57ec"}
Jul 23 00:43:52.393939 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:52.393856    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" podStartSLOduration=1.393822307 podStartE2EDuration="1.393822307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:43:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:43:52.392512438 +0000 UTC m=+2382.116827624" watchObservedRunningTime="2026-07-23 00:43:52.393822307 +0000 UTC m=+2382.118137491"
Jul 23 00:43:53.125214 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:53.125181    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:53.130008 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:53.129980    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:53.379405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:53.379329    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:43:53.380186 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:43:53.380167    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:44:06.461803 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:06.461726    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerID="23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11" exitCode=0
Jul 23 00:44:06.462219 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:06.461799    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" event={"ID":"e1dbf888-3814-49cb-8baf-c856b9282bff","Type":"ContainerDied","Data":"23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"}
Jul 23 00:44:06.462219 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:06.462188    2558 scope.go:117] "RemoveContainer" containerID="23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"
Jul 23 00:44:06.463256 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:06.463230    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:44:06.463335 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:06.463295    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:44:06.463335 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:06.463326    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:44:07.469361 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:07.469322    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" event={"ID":"e1dbf888-3814-49cb-8baf-c856b9282bff","Type":"ContainerStarted","Data":"b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548"}
Jul 23 00:44:08.124581 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:08.124548    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:44:08.125773 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:08.125746    2558 patch_prober.go:28] interesting pod/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 23 00:44:08.125912 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:08.125794    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 23 00:44:08.379274 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:08.379176    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:44:09.125146 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:09.125101    2558 patch_prober.go:28] interesting pod/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 23 00:44:09.125557 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:09.125159    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 23 00:44:10.124689 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:10.124656    2558 patch_prober.go:28] interesting pod/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 23 00:44:10.124882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:10.124712    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 23 00:44:10.912858 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:10.912816    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:44:10.920847 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:10.920806    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:44:11.016111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:11.016083    2558 scope.go:117] "RemoveContainer" containerID="208479972a37e8a00e13e9b775eaf5c1ccc7fefb81a4ec18d0877dd373bfc813"
Jul 23 00:44:11.125014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:11.124981    2558 patch_prober.go:28] interesting pod/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused" start-of-body=
Jul 23 00:44:11.125178 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:11.125045    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.68:15021/healthz/ready\": dial tcp 10.132.0.68:15021: connect: connection refused"
Jul 23 00:44:12.128643 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:12.128613    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:44:12.129533 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:12.129513    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:44:54.174086 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.174012    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"]
Jul 23 00:44:54.177634 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.177615    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.183767 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.183742    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2f7578\"/\"openshift-service-ca.crt\""
Jul 23 00:44:54.183924 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.183806    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2f7578\"/\"test-model-2f7578-kserve-self-signed-certs\""
Jul 23 00:44:54.183924 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.183817    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-2f7578\"/\"default-dockercfg-7s2hc\""
Jul 23 00:44:54.184056 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.183926    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-2f7578\"/\"kube-root-ca.crt\""
Jul 23 00:44:54.196877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.196854    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"]
Jul 23 00:44:54.342623 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.342589    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-dshm\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.342623 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.342620    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-home\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.342817 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.342648    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb68256-9c50-4e68-9296-1612d36f4fc1-tls-certs\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.342817 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.342700    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-model-cache\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.342817 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.342753    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-tmp-dir\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.342817 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.342791    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8ngw\" (UniqueName: \"kubernetes.io/projected/efb68256-9c50-4e68-9296-1612d36f4fc1-kube-api-access-w8ngw\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444033 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.443957    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8ngw\" (UniqueName: \"kubernetes.io/projected/efb68256-9c50-4e68-9296-1612d36f4fc1-kube-api-access-w8ngw\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444033 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444003    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-dshm\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444033 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444023    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-home\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444298 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444057    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb68256-9c50-4e68-9296-1612d36f4fc1-tls-certs\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444298 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444095    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-model-cache\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444298 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444174    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-tmp-dir\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444461 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444428    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-home\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444546 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444525    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-model-cache\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.444602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.444554    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-tmp-dir\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.446334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.446305    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-dshm\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.446531 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.446515    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb68256-9c50-4e68-9296-1612d36f4fc1-tls-certs\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.453215 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.453194    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8ngw\" (UniqueName: \"kubernetes.io/projected/efb68256-9c50-4e68-9296-1612d36f4fc1-kube-api-access-w8ngw\") pod \"test-model-2f7578-kserve-846ddd8479-ctp7z\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") " pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.487345 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.487325    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:44:54.629325 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.629295    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"]
Jul 23 00:44:54.631103 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:44:54.631071    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podefb68256_9c50_4e68_9296_1612d36f4fc1.slice/crio-f3bc4a24bea55b33f9df8b8433c2d764efbb925a252cdab25351671af29ce8c1 WatchSource:0}: Error finding container f3bc4a24bea55b33f9df8b8433c2d764efbb925a252cdab25351671af29ce8c1: Status 404 returned error can't find the container with id f3bc4a24bea55b33f9df8b8433c2d764efbb925a252cdab25351671af29ce8c1
Jul 23 00:44:54.769893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.769805    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z" event={"ID":"efb68256-9c50-4e68-9296-1612d36f4fc1","Type":"ContainerStarted","Data":"4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26"}
Jul 23 00:44:54.769893 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.769866    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z" event={"ID":"efb68256-9c50-4e68-9296-1612d36f4fc1","Type":"ContainerStarted","Data":"f3bc4a24bea55b33f9df8b8433c2d764efbb925a252cdab25351671af29ce8c1"}
Jul 23 00:44:54.791037 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:44:54.790993    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z" podStartSLOduration=0.790976993 podStartE2EDuration="790.976993ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:44:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:44:54.789958813 +0000 UTC m=+2444.514274017" watchObservedRunningTime="2026-07-23 00:44:54.790976993 +0000 UTC m=+2444.515292382"
Jul 23 00:45:00.139403 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.139370    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"]
Jul 23 00:45:00.142970 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.142952    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"
Jul 23 00:45:00.145566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.145542    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:45:00.154543 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.154517    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"
Jul 23 00:45:00.154727 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.154704    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"]
Jul 23 00:45:00.304069 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.304036    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"]
Jul 23 00:45:00.305001 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:45:00.304971    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c89745_d836_48fa_97db_280010024632.slice/crio-39e78c90bcf8091979a42f198d1d05732ed50e6035493eceb6a06562881ab784 WatchSource:0}: Error finding container 39e78c90bcf8091979a42f198d1d05732ed50e6035493eceb6a06562881ab784: Status 404 returned error can't find the container with id 39e78c90bcf8091979a42f198d1d05732ed50e6035493eceb6a06562881ab784
Jul 23 00:45:00.806190 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.806105    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl" event={"ID":"e7c89745-d836-48fa-97db-280010024632","Type":"ContainerStarted","Data":"0bd8b7c3238e075114aea4eaac9b1403a95362a1e7d3842dc1a809d5eabce376"}
Jul 23 00:45:00.806190 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.806162    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl" event={"ID":"e7c89745-d836-48fa-97db-280010024632","Type":"ContainerStarted","Data":"39e78c90bcf8091979a42f198d1d05732ed50e6035493eceb6a06562881ab784"}
Jul 23 00:45:00.832509 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:00.832461    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl" podStartSLOduration=0.832446916 podStartE2EDuration="832.446916ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:45:00.830284236 +0000 UTC m=+2450.554599419" watchObservedRunningTime="2026-07-23 00:45:00.832446916 +0000 UTC m=+2450.556762132"
Jul 23 00:45:01.812813 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:01.812781    2558 generic.go:356] "Generic (PLEG): container finished" podID="e7c89745-d836-48fa-97db-280010024632" containerID="0bd8b7c3238e075114aea4eaac9b1403a95362a1e7d3842dc1a809d5eabce376" exitCode=0
Jul 23 00:45:01.813220 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:01.812862    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl" event={"ID":"e7c89745-d836-48fa-97db-280010024632","Type":"ContainerDied","Data":"0bd8b7c3238e075114aea4eaac9b1403a95362a1e7d3842dc1a809d5eabce376"}
Jul 23 00:45:02.953502 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:02.953478    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"
Jul 23 00:45:03.825729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:03.825692    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl" event={"ID":"e7c89745-d836-48fa-97db-280010024632","Type":"ContainerDied","Data":"39e78c90bcf8091979a42f198d1d05732ed50e6035493eceb6a06562881ab784"}
Jul 23 00:45:03.825729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:03.825727    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"
Jul 23 00:45:03.825957 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:03.825739    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39e78c90bcf8091979a42f198d1d05732ed50e6035493eceb6a06562881ab784"
Jul 23 00:45:04.488370 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:04.488337    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:45:04.488748 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:04.488390    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:45:04.500882 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:04.500855    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:45:04.842379 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:04.842344    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:45:50.597610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:45:50.597575    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"]
Jul 23 00:46:05.617282 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.617181    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z" podUID="efb68256-9c50-4e68-9296-1612d36f4fc1" containerName="main" containerID="cri-o://4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26" gracePeriod=45
Jul 23 00:46:05.878171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.878117    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:46:05.968552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968526    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb68256-9c50-4e68-9296-1612d36f4fc1-tls-certs\") pod \"efb68256-9c50-4e68-9296-1612d36f4fc1\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") "
Jul 23 00:46:05.968705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968579    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w8ngw\" (UniqueName: \"kubernetes.io/projected/efb68256-9c50-4e68-9296-1612d36f4fc1-kube-api-access-w8ngw\") pod \"efb68256-9c50-4e68-9296-1612d36f4fc1\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") "
Jul 23 00:46:05.968705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968619    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-dshm\") pod \"efb68256-9c50-4e68-9296-1612d36f4fc1\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") "
Jul 23 00:46:05.968705 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968671    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-home\") pod \"efb68256-9c50-4e68-9296-1612d36f4fc1\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") "
Jul 23 00:46:05.968915 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968801    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-tmp-dir\") pod \"efb68256-9c50-4e68-9296-1612d36f4fc1\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") "
Jul 23 00:46:05.968915 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968894    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-model-cache\") pod \"efb68256-9c50-4e68-9296-1612d36f4fc1\" (UID: \"efb68256-9c50-4e68-9296-1612d36f4fc1\") "
Jul 23 00:46:05.969089 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.968938    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-home" (OuterVolumeSpecName: "home") pod "efb68256-9c50-4e68-9296-1612d36f4fc1" (UID: "efb68256-9c50-4e68-9296-1612d36f4fc1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:05.969089 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.969036    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "efb68256-9c50-4e68-9296-1612d36f4fc1" (UID: "efb68256-9c50-4e68-9296-1612d36f4fc1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:05.969228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.969205    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-model-cache" (OuterVolumeSpecName: "model-cache") pod "efb68256-9c50-4e68-9296-1612d36f4fc1" (UID: "efb68256-9c50-4e68-9296-1612d36f4fc1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:05.969318 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.969303    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:05.969376 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.969324    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:05.969376 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.969340    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:05.971133 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.971099    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efb68256-9c50-4e68-9296-1612d36f4fc1-kube-api-access-w8ngw" (OuterVolumeSpecName: "kube-api-access-w8ngw") pod "efb68256-9c50-4e68-9296-1612d36f4fc1" (UID: "efb68256-9c50-4e68-9296-1612d36f4fc1"). InnerVolumeSpecName "kube-api-access-w8ngw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:46:05.971237 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.971141    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efb68256-9c50-4e68-9296-1612d36f4fc1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "efb68256-9c50-4e68-9296-1612d36f4fc1" (UID: "efb68256-9c50-4e68-9296-1612d36f4fc1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:46:05.971323 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:05.971307    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-dshm" (OuterVolumeSpecName: "dshm") pod "efb68256-9c50-4e68-9296-1612d36f4fc1" (UID: "efb68256-9c50-4e68-9296-1612d36f4fc1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:06.070252 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.070226    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/efb68256-9c50-4e68-9296-1612d36f4fc1-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:06.070344 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.070256    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w8ngw\" (UniqueName: \"kubernetes.io/projected/efb68256-9c50-4e68-9296-1612d36f4fc1-kube-api-access-w8ngw\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:06.070344 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.070271    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/efb68256-9c50-4e68-9296-1612d36f4fc1-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:06.194928 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.194869    2558 generic.go:356] "Generic (PLEG): container finished" podID="efb68256-9c50-4e68-9296-1612d36f4fc1" containerID="4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26" exitCode=0
Jul 23 00:46:06.194928 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.194908    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"
Jul 23 00:46:06.195052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.194953    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z" event={"ID":"efb68256-9c50-4e68-9296-1612d36f4fc1","Type":"ContainerDied","Data":"4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26"}
Jul 23 00:46:06.195052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.194996    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z" event={"ID":"efb68256-9c50-4e68-9296-1612d36f4fc1","Type":"ContainerDied","Data":"f3bc4a24bea55b33f9df8b8433c2d764efbb925a252cdab25351671af29ce8c1"}
Jul 23 00:46:06.195052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.195013    2558 scope.go:117] "RemoveContainer" containerID="4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26"
Jul 23 00:46:06.206767 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.206750    2558 scope.go:117] "RemoveContainer" containerID="4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26"
Jul 23 00:46:06.207043 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:46:06.207027    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26\": container with ID starting with 4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26 not found: ID does not exist" containerID="4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26"
Jul 23 00:46:06.207098 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.207049    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26"} err="failed to get container status \"4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26\": rpc error: code = NotFound desc = could not find container \"4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26\": container with ID starting with 4e7fbf57799abc204e9dc10abdb04decff1c06206e0e5e587bb4ba505f68ee26 not found: ID does not exist"
Jul 23 00:46:06.219898 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.219877    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"]
Jul 23 00:46:06.225356 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.225336    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-2f7578/test-model-2f7578-kserve-846ddd8479-ctp7z"]
Jul 23 00:46:06.735891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:06.735861    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efb68256-9c50-4e68-9296-1612d36f4fc1" path="/var/lib/kubelet/pods/efb68256-9c50-4e68-9296-1612d36f4fc1/volumes"
Jul 23 00:46:11.046027 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:11.045996    2558 scope.go:117] "RemoveContainer" containerID="256aa0f6efcd23f66b56cd9a79694db9133593c0bbca1db9d7125fcee3940021"
Jul 23 00:46:23.433928 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:23.433893    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"]
Jul 23 00:46:23.437627 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:23.437603    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-inf-b-2f7578-29746125-8bskl"]
Jul 23 00:46:24.735812 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:24.735775    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7c89745-d836-48fa-97db-280010024632" path="/var/lib/kubelet/pods/e7c89745-d836-48fa-97db-280010024632/volumes"
Jul 23 00:46:34.841926 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:34.841895    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"]
Jul 23 00:46:34.842299 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:34.842091    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy" containerID="cri-o://b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548" gracePeriod=30
Jul 23 00:46:40.107999 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.107943    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:46:40.259237 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259205    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-credential-socket\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259419 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259245    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1dbf888-3814-49cb-8baf-c856b9282bff-istiod-ca-cert\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259419 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259266    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-socket\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259419 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259396    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-data\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259442    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw9j4\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-kube-api-access-jw9j4\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259489    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-certs\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259507    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-token\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259515    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:40.259591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259536    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-envoy\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259591 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259559    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:40.259914 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259607    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-podinfo\") pod \"e1dbf888-3814-49cb-8baf-c856b9282bff\" (UID: \"e1dbf888-3814-49cb-8baf-c856b9282bff\") "
Jul 23 00:46:40.259914 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259667    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e1dbf888-3814-49cb-8baf-c856b9282bff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:46:40.259914 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.259824    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:40.260073 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.260014    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-socket\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.260073 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.260036    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-workload-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.260073 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.260052    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-credential-socket\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.260073 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.260066    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e1dbf888-3814-49cb-8baf-c856b9282bff-istiod-ca-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.260465 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.260441    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-data" (OuterVolumeSpecName: "istio-data") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:40.261918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.261885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:46:40.262027 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.261921    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:46:40.262131 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.262113    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-token" (OuterVolumeSpecName: "istio-token") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:46:40.262176 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.262127    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-kube-api-access-jw9j4" (OuterVolumeSpecName: "kube-api-access-jw9j4") pod "e1dbf888-3814-49cb-8baf-c856b9282bff" (UID: "e1dbf888-3814-49cb-8baf-c856b9282bff"). InnerVolumeSpecName "kube-api-access-jw9j4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:46:40.360734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.360680    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-data\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.360734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.360702    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw9j4\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-kube-api-access-jw9j4\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.360734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.360713    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-token\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.360734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.360721    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-envoy\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.360734 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.360730    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e1dbf888-3814-49cb-8baf-c856b9282bff-istio-podinfo\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:46:40.396810 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.396779    2558 generic.go:356] "Generic (PLEG): container finished" podID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerID="b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548" exitCode=0
Jul 23 00:46:40.396931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.396825    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"
Jul 23 00:46:40.396931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.396874    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" event={"ID":"e1dbf888-3814-49cb-8baf-c856b9282bff","Type":"ContainerDied","Data":"b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548"}
Jul 23 00:46:40.397014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.396957    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g" event={"ID":"e1dbf888-3814-49cb-8baf-c856b9282bff","Type":"ContainerDied","Data":"20529eed338e98157a305e6150333ef72f1821fab0a697c29e0791cecc1d57ec"}
Jul 23 00:46:40.397014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.396983    2558 scope.go:117] "RemoveContainer" containerID="b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548"
Jul 23 00:46:40.406332 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.406314    2558 scope.go:117] "RemoveContainer" containerID="23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"
Jul 23 00:46:40.414064 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.414047    2558 scope.go:117] "RemoveContainer" containerID="b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548"
Jul 23 00:46:40.414300 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:46:40.414280    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548\": container with ID starting with b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548 not found: ID does not exist" containerID="b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548"
Jul 23 00:46:40.414352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.414307    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548"} err="failed to get container status \"b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548\": rpc error: code = NotFound desc = could not find container \"b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548\": container with ID starting with b5c2b73aaaceca3d7ab8e25aef37f7b10a8601c08cfd031bda5a3f49bbf58548 not found: ID does not exist"
Jul 23 00:46:40.414352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.414327    2558 scope.go:117] "RemoveContainer" containerID="23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"
Jul 23 00:46:40.414575 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:46:40.414559    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11\": container with ID starting with 23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11 not found: ID does not exist" containerID="23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"
Jul 23 00:46:40.414632 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.414583    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11"} err="failed to get container status \"23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11\": rpc error: code = NotFound desc = could not find container \"23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11\": container with ID starting with 23ef0c094cb3376935a3079c2742a7f4031635977ae913fcc1df567993d0da11 not found: ID does not exist"
Jul 23 00:46:40.559555 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.559530    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"]
Jul 23 00:46:40.579769 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.579748    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2f7578-openshift-default-556fd8b55c-zcd6g"]
Jul 23 00:46:40.735746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:40.735686    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" path="/var/lib/kubelet/pods/e1dbf888-3814-49cb-8baf-c856b9282bff/volumes"
Jul 23 00:46:44.661452 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661414    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"]
Jul 23 00:46:44.661818 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661799    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy"
Jul 23 00:46:44.661818 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661810    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy"
Jul 23 00:46:44.661919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661822    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7c89745-d836-48fa-97db-280010024632" containerName="cleanup"
Jul 23 00:46:44.661919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661829    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7c89745-d836-48fa-97db-280010024632" containerName="cleanup"
Jul 23 00:46:44.661919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661868    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efb68256-9c50-4e68-9296-1612d36f4fc1" containerName="main"
Jul 23 00:46:44.661919 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661875    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="efb68256-9c50-4e68-9296-1612d36f4fc1" containerName="main"
Jul 23 00:46:44.662042 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661935    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="efb68256-9c50-4e68-9296-1612d36f4fc1" containerName="main"
Jul 23 00:46:44.662042 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661946    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy"
Jul 23 00:46:44.662042 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661955    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7c89745-d836-48fa-97db-280010024632" containerName="cleanup"
Jul 23 00:46:44.662042 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.661961    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy"
Jul 23 00:46:44.666352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.666333    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.669730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.669712    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-00cf30-openshift-default-dockercfg-kgdxf\""
Jul 23 00:46:44.683741 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.683721    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"]
Jul 23 00:46:44.795566 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795537    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-credential-socket\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795581    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-envoy\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795601    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-data\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795681    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-certs\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795717    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7496c2-2068-46ee-826e-3d7fdff903d5-istiod-ca-cert\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795752    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-podinfo\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795810    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-token\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795830    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf9rq\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-kube-api-access-kf9rq\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.795923 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.795899    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-socket\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.896889 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.896828    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-podinfo\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.896912    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-token\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.896941    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf9rq\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-kube-api-access-kf9rq\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897029 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.896987    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-socket\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897043    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-credential-socket\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897096    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-envoy\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897124    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-data\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897250    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-certs\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897473 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897289    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7496c2-2068-46ee-826e-3d7fdff903d5-istiod-ca-cert\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897529 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897477    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-credential-socket\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897585 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897552    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-certs\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897649 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897601    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-data\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.897752 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.897733    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-socket\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.898084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.898062    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7496c2-2068-46ee-826e-3d7fdff903d5-istiod-ca-cert\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.899310 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.899286    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-envoy\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.899409 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.899385    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-podinfo\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.907746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.907723    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-token\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.910508 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.910485    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf9rq\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-kube-api-access-kf9rq\") pod \"e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") " pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:44.978621 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:44.978566    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:45.149312 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.149285    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"]
Jul 23 00:46:45.150919 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:46:45.150891    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c7496c2_2068_46ee_826e_3d7fdff903d5.slice/crio-7d2ca8dad954874fcf4a2409b92d19fa2b5c948c55ccacfca77e0ec118ebd2b2 WatchSource:0}: Error finding container 7d2ca8dad954874fcf4a2409b92d19fa2b5c948c55ccacfca77e0ec118ebd2b2: Status 404 returned error can't find the container with id 7d2ca8dad954874fcf4a2409b92d19fa2b5c948c55ccacfca77e0ec118ebd2b2
Jul 23 00:46:45.152994 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.152962    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:46:45.153085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.153026    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:46:45.153085 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.153056    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:46:45.430342 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.430304    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" event={"ID":"8c7496c2-2068-46ee-826e-3d7fdff903d5","Type":"ContainerStarted","Data":"d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"}
Jul 23 00:46:45.430630 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.430361    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" event={"ID":"8c7496c2-2068-46ee-826e-3d7fdff903d5","Type":"ContainerStarted","Data":"7d2ca8dad954874fcf4a2409b92d19fa2b5c948c55ccacfca77e0ec118ebd2b2"}
Jul 23 00:46:45.457596 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.457548    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podStartSLOduration=1.4575328189999999 podStartE2EDuration="1.457532819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:46:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:46:45.455157017 +0000 UTC m=+2555.179472204" watchObservedRunningTime="2026-07-23 00:46:45.457532819 +0000 UTC m=+2555.181848003"
Jul 23 00:46:45.979612 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:45.979579    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:46.983484 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:46.983454    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:47.442716 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:47.442682    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:46:47.443679 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:46:47.443660    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:47:03.357453 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.356649    2558 patch_prober.go:28] interesting pod/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": read tcp 10.132.0.2:58822->10.132.0.71:15021: read: connection reset by peer" start-of-body=
Jul 23 00:47:03.357453 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.356727    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": read tcp 10.132.0.2:58822->10.132.0.71:15021: read: connection reset by peer"
Jul 23 00:47:03.541534 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.541460    2558 generic.go:356] "Generic (PLEG): container finished" podID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerID="d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504" exitCode=0
Jul 23 00:47:03.541534 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.541499    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" event={"ID":"8c7496c2-2068-46ee-826e-3d7fdff903d5","Type":"ContainerDied","Data":"d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"}
Jul 23 00:47:03.541911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.541896    2558 scope.go:117] "RemoveContainer" containerID="d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"
Jul 23 00:47:03.542894 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.542862    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:47:03.542990 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.542931    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:47:03.542990 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:03.542966    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:47:04.549246 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:04.549215    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" event={"ID":"8c7496c2-2068-46ee-826e-3d7fdff903d5","Type":"ContainerStarted","Data":"c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84"}
Jul 23 00:47:04.979383 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:04.979345    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:47:04.979590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:04.979465    2558 patch_prober.go:28] interesting pod/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 23 00:47:04.979590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:04.979509    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 23 00:47:05.979575 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:05.979539    2558 patch_prober.go:28] interesting pod/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 23 00:47:05.980172 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:05.980141    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 23 00:47:06.985229 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:06.985192    2558 patch_prober.go:28] interesting pod/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 23 00:47:06.985790 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:06.985745    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 23 00:47:07.979888 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:07.979829    2558 patch_prober.go:28] interesting pod/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Jul 23 00:47:07.980203 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:07.980172    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Jul 23 00:47:08.982344 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:08.982315    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:47:08.982782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:08.982722    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:47:08.983359 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:08.983341    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:47:55.301046 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:55.301007    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"]
Jul 23 00:47:55.301447 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:47:55.301285    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy" containerID="cri-o://c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84" gracePeriod=30
Jul 23 00:48:00.553012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.552989    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:48:00.657546 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657486    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-data\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657665 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657557    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-podinfo\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657665 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657582    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7496c2-2068-46ee-826e-3d7fdff903d5-istiod-ca-cert\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657665 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657605    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-token\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657665 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657626    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-certs\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657665 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657649    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-envoy\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657683    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-socket\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657744    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-credential-socket\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.657953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657786    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf9rq\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-kube-api-access-kf9rq\") pod \"8c7496c2-2068-46ee-826e-3d7fdff903d5\" (UID: \"8c7496c2-2068-46ee-826e-3d7fdff903d5\") "
Jul 23 00:48:00.658126 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.657941    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:48:00.658126 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.658092    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:48:00.658293 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.658271    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:48:00.658365 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.658312    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:48:00.658433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.658384    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8c7496c2-2068-46ee-826e-3d7fdff903d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:48:00.658433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.658424    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.658538 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.658445    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-workload-socket\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.659939 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.659908    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:48:00.659939 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.659926    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:48:00.660266 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.660251    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-kube-api-access-kf9rq" (OuterVolumeSpecName: "kube-api-access-kf9rq") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "kube-api-access-kf9rq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:48:00.660322 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.660282    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "8c7496c2-2068-46ee-826e-3d7fdff903d5" (UID: "8c7496c2-2068-46ee-826e-3d7fdff903d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:48:00.759311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759282    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-podinfo\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.759311 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759308    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8c7496c2-2068-46ee-826e-3d7fdff903d5-istiod-ca-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.759456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759316    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-token\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.759456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759324    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-envoy\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.759456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759331    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-credential-socket\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.759456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759340    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf9rq\" (UniqueName: \"kubernetes.io/projected/8c7496c2-2068-46ee-826e-3d7fdff903d5-kube-api-access-kf9rq\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.759456 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.759348    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8c7496c2-2068-46ee-826e-3d7fdff903d5-istio-data\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:48:00.870844 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.870809    2558 generic.go:356] "Generic (PLEG): container finished" podID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerID="c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84" exitCode=0
Jul 23 00:48:00.870995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.870905    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"
Jul 23 00:48:00.870995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.870906    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" event={"ID":"8c7496c2-2068-46ee-826e-3d7fdff903d5","Type":"ContainerDied","Data":"c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84"}
Jul 23 00:48:00.870995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.870970    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7" event={"ID":"8c7496c2-2068-46ee-826e-3d7fdff903d5","Type":"ContainerDied","Data":"7d2ca8dad954874fcf4a2409b92d19fa2b5c948c55ccacfca77e0ec118ebd2b2"}
Jul 23 00:48:00.871177 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.871000    2558 scope.go:117] "RemoveContainer" containerID="c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84"
Jul 23 00:48:00.880941 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.880913    2558 scope.go:117] "RemoveContainer" containerID="d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"
Jul 23 00:48:00.889587 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.889571    2558 scope.go:117] "RemoveContainer" containerID="c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84"
Jul 23 00:48:00.889824 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:48:00.889798    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84\": container with ID starting with c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84 not found: ID does not exist" containerID="c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84"
Jul 23 00:48:00.889922 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.889822    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84"} err="failed to get container status \"c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84\": rpc error: code = NotFound desc = could not find container \"c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84\": container with ID starting with c100499e0cc55306dd8a78037d5901e35310fe0879c4ab211920998a84903d84 not found: ID does not exist"
Jul 23 00:48:00.889922 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.889863    2558 scope.go:117] "RemoveContainer" containerID="d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"
Jul 23 00:48:00.890079 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:48:00.890059    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504\": container with ID starting with d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504 not found: ID does not exist" containerID="d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"
Jul 23 00:48:00.890148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.890091    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504"} err="failed to get container status \"d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504\": rpc error: code = NotFound desc = could not find container \"d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504\": container with ID starting with d23b0451415047c47bb2f0ac2182397e79773dc15c7c577fa6fed7e196195504 not found: ID does not exist"
Jul 23 00:48:00.893806 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.893773    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"]
Jul 23 00:48:00.901848 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:00.901820    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-00cf30-openshift-default-5c7b5cf99c-stgl7"]
Jul 23 00:48:02.735027 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:02.734994    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" path="/var/lib/kubelet/pods/8c7496c2-2068-46ee-826e-3d7fdff903d5/volumes"
Jul 23 00:48:04.957774 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.957735    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"]
Jul 23 00:48:04.958327 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958308    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy"
Jul 23 00:48:04.958424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958330    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy"
Jul 23 00:48:04.958424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958346    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy"
Jul 23 00:48:04.958424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958354    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy"
Jul 23 00:48:04.958424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958380    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy"
Jul 23 00:48:04.958424 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958389    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1dbf888-3814-49cb-8baf-c856b9282bff" containerName="istio-proxy"
Jul 23 00:48:04.958666 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958533    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy"
Jul 23 00:48:04.958666 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.958547    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8c7496c2-2068-46ee-826e-3d7fdff903d5" containerName="istio-proxy"
Jul 23 00:48:04.963086 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.963065    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.966321 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.966301    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-d50941-openshift-default-dockercfg-km2tv\""
Jul 23 00:48:04.980563 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.980543    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"]
Jul 23 00:48:04.993818 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.993796    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-data\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.993932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.993846    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-istio-token\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.993932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.993879    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-envoy\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.993932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.993915    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-credential-socket\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.994041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.993942    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-certs\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.994041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.993976    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8affd781-851e-4997-b862-08dedd2e8479-istio-podinfo\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.994041 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.994031    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8pjj\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-kube-api-access-v8pjj\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.994150 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.994047    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8affd781-851e-4997-b862-08dedd2e8479-istiod-ca-cert\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:04.994150 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:04.994065    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-socket\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095288 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095255    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-data\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095425 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095306    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-istio-token\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095425 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095336    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-envoy\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095425 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095379    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-credential-socket\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095496    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-certs\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095590 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095533    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8affd781-851e-4997-b862-08dedd2e8479-istio-podinfo\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095605    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8pjj\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-kube-api-access-v8pjj\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095639    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8affd781-851e-4997-b862-08dedd2e8479-istiod-ca-cert\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095684    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-data\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095916 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095813    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-socket\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095916 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095855    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-credential-socket\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.095916 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.095866    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-certs\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.096157 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.096138    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-socket\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.096284 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.096262    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8affd781-851e-4997-b862-08dedd2e8479-istiod-ca-cert\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.097683 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.097664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-envoy\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.098002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.097984    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8affd781-851e-4997-b862-08dedd2e8479-istio-podinfo\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.104579 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.104558    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-istio-token\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.104730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.104712    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8pjj\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-kube-api-access-v8pjj\") pod \"e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") " pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.274863 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.274781    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:05.415911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.415886    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"]
Jul 23 00:48:05.416660 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:48:05.416636    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8affd781_851e_4997_b862_08dedd2e8479.slice/crio-a43456127289e0acbaa99900355c0756d8eefb12818d7f6447f3346abd7506f5 WatchSource:0}: Error finding container a43456127289e0acbaa99900355c0756d8eefb12818d7f6447f3346abd7506f5: Status 404 returned error can't find the container with id a43456127289e0acbaa99900355c0756d8eefb12818d7f6447f3346abd7506f5
Jul 23 00:48:05.418851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.418809    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:48:05.418947 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.418898    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:48:05.418989 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.418942    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:48:05.901902 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.901855    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" event={"ID":"8affd781-851e-4997-b862-08dedd2e8479","Type":"ContainerStarted","Data":"d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"}
Jul 23 00:48:05.902212 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.901918    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" event={"ID":"8affd781-851e-4997-b862-08dedd2e8479","Type":"ContainerStarted","Data":"a43456127289e0acbaa99900355c0756d8eefb12818d7f6447f3346abd7506f5"}
Jul 23 00:48:05.926003 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:05.925949    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podStartSLOduration=1.925931308 podStartE2EDuration="1.925931308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:48:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:48:05.923785484 +0000 UTC m=+2635.648100668" watchObservedRunningTime="2026-07-23 00:48:05.925931308 +0000 UTC m=+2635.650246493"
Jul 23 00:48:06.275714 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:06.275640    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:06.280738 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:06.280711    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:06.908433 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:06.908404    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:06.909339 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:06.909320    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:21.910485 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:21.910453    2558 patch_prober.go:28] interesting pod/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 23 00:48:21.910823 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:21.910511    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 23 00:48:22.004949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:22.004869    2558 generic.go:356] "Generic (PLEG): container finished" podID="8affd781-851e-4997-b862-08dedd2e8479" containerID="d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c" exitCode=0
Jul 23 00:48:22.004949 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:22.004931    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" event={"ID":"8affd781-851e-4997-b862-08dedd2e8479","Type":"ContainerDied","Data":"d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"}
Jul 23 00:48:22.005338 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:22.005323    2558 scope.go:117] "RemoveContainer" containerID="d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"
Jul 23 00:48:22.006394 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:22.006359    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:48:22.006461 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:22.006432    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:48:22.006503 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:22.006468    2558 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"}
Jul 23 00:48:23.012920 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:23.012886    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" event={"ID":"8affd781-851e-4997-b862-08dedd2e8479","Type":"ContainerStarted","Data":"918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2"}
Jul 23 00:48:23.275641 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:23.275550    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:23.275818 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:23.275693    2558 patch_prober.go:28] interesting pod/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 23 00:48:23.275818 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:23.275739    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 23 00:48:24.284868 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:24.275225    2558 patch_prober.go:28] interesting pod/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 23 00:48:24.284868 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:24.275285    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 23 00:48:25.275297 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:25.275252    2558 patch_prober.go:28] interesting pod/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 23 00:48:25.275547 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:25.275318    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 23 00:48:26.276235 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:26.276197    2558 patch_prober.go:28] interesting pod/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body=
Jul 23 00:48:26.276781 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:26.276749    2558 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused"
Jul 23 00:48:27.279030 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:27.279000    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:27.279507 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:27.279375    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:27.279992 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:27.279974    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:48:43.069873 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.069817    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"]
Jul 23 00:48:43.073786 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.073767    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.078483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.078463    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"auth-test-model-d50941-kserve-self-signed-certs\""
Jul 23 00:48:43.078483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.078486    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"kube-root-ca.crt\""
Jul 23 00:48:43.078483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.078468    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"default-dockercfg-69j6c\""
Jul 23 00:48:43.078483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.078523    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"openshift-service-ca.crt\""
Jul 23 00:48:43.089224 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.089206    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"]
Jul 23 00:48:43.135304 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.135283    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzvzk\" (UniqueName: \"kubernetes.io/projected/358394b7-2288-4fab-9cf4-7ab9a761bc45-kube-api-access-dzvzk\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.135406 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.135327    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-home\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.135406 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.135347    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-tmp-dir\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.135496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.135441    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-dshm\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.135496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.135467    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-model-cache\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.135496 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.135490    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358394b7-2288-4fab-9cf4-7ab9a761bc45-tls-certs\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.236897 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.236870    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-dshm\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.236902    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-model-cache\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.236924    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358394b7-2288-4fab-9cf4-7ab9a761bc45-tls-certs\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.236952    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzvzk\" (UniqueName: \"kubernetes.io/projected/358394b7-2288-4fab-9cf4-7ab9a761bc45-kube-api-access-dzvzk\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.236992    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-home\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237047 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.237012    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-tmp-dir\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237358 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.237338    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-model-cache\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.237370    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-home\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.237408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.237398    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-tmp-dir\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.239201 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.239172    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-dshm\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.239429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.239409    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358394b7-2288-4fab-9cf4-7ab9a761bc45-tls-certs\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.245202 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.245183    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzvzk\" (UniqueName: \"kubernetes.io/projected/358394b7-2288-4fab-9cf4-7ab9a761bc45-kube-api-access-dzvzk\") pod \"auth-test-model-d50941-kserve-6777974947-2d7b4\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") " pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.384074 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.383984    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:43.521846 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:43.521804    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"]
Jul 23 00:48:43.522713 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:48:43.522676    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod358394b7_2288_4fab_9cf4_7ab9a761bc45.slice/crio-30004e9af469f1a2257741ea8c559e509eed6cb59b3d17fc6cbe6440d7782fe9 WatchSource:0}: Error finding container 30004e9af469f1a2257741ea8c559e509eed6cb59b3d17fc6cbe6440d7782fe9: Status 404 returned error can't find the container with id 30004e9af469f1a2257741ea8c559e509eed6cb59b3d17fc6cbe6440d7782fe9
Jul 23 00:48:44.140610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:44.140572    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4" event={"ID":"358394b7-2288-4fab-9cf4-7ab9a761bc45","Type":"ContainerStarted","Data":"ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73"}
Jul 23 00:48:44.140610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:44.140614    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4" event={"ID":"358394b7-2288-4fab-9cf4-7ab9a761bc45","Type":"ContainerStarted","Data":"30004e9af469f1a2257741ea8c559e509eed6cb59b3d17fc6cbe6440d7782fe9"}
Jul 23 00:48:44.175118 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:44.175068    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4" podStartSLOduration=1.17505331 podStartE2EDuration="1.17505331s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:48:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:48:44.173671676 +0000 UTC m=+2673.897986859" watchObservedRunningTime="2026-07-23 00:48:44.17505331 +0000 UTC m=+2673.899368494"
Jul 23 00:48:53.384667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:53.384631    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:53.385206 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:53.384685    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:53.397279 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:53.397256    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:48:54.209021 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:48:54.208984    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:49:10.952531 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:49:10.952450    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:49:10.959931 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:49:10.959909    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:50:46.279256 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:46.279220    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"]
Jul 23 00:50:59.429171 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.429136    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"]
Jul 23 00:50:59.432788 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.432765    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.436253 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.436232    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-59a77d\"/\"sub-test-model-59a77d-kserve-self-signed-certs\""
Jul 23 00:50:59.445094 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.445066    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-59a77d\"/\"openshift-service-ca.crt\""
Jul 23 00:50:59.445374 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.445353    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-59a77d\"/\"kube-root-ca.crt\""
Jul 23 00:50:59.446375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.446350    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-59a77d\"/\"default-dockercfg-lh9xc\""
Jul 23 00:50:59.448324 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.448299    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"]
Jul 23 00:50:59.486476 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.486444    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tls-certs\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.486990 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.486969    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tmp-dir\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.487156 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.487133    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-dshm\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.487268 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.487253    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-model-cache\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.487386 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.487371    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pjgx\" (UniqueName: \"kubernetes.io/projected/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-kube-api-access-7pjgx\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.487499 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.487485    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-home\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588255 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588220    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tmp-dir\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588271    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-dshm\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588289    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-model-cache\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588320    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pjgx\" (UniqueName: \"kubernetes.io/projected/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-kube-api-access-7pjgx\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588352    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-home\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588431    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tls-certs\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588738 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588714    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-model-cache\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588807 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588759    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tmp-dir\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.588975 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.588956    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-home\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.591139 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.591120    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-dshm\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.591383 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.591366    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tls-certs\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.600054 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.600034    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pjgx\" (UniqueName: \"kubernetes.io/projected/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-kube-api-access-7pjgx\") pod \"sub-test-model-59a77d-kserve-66469d54f7-psnz2\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") " pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.742625 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.742533    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:50:59.888040 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.888015    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"]
Jul 23 00:50:59.888880 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:50:59.888829    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod68ebd4be_bd96_4d2a_b4c3_44ba43f4eb97.slice/crio-ee164248b58aee03738b5e2bd24091923aa65d811dd559873406ebe31f52fc27 WatchSource:0}: Error finding container ee164248b58aee03738b5e2bd24091923aa65d811dd559873406ebe31f52fc27: Status 404 returned error can't find the container with id ee164248b58aee03738b5e2bd24091923aa65d811dd559873406ebe31f52fc27
Jul 23 00:50:59.914753 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:50:59.914730    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2" event={"ID":"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97","Type":"ContainerStarted","Data":"ee164248b58aee03738b5e2bd24091923aa65d811dd559873406ebe31f52fc27"}
Jul 23 00:51:00.923451 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:00.923414    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2" event={"ID":"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97","Type":"ContainerStarted","Data":"fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727"}
Jul 23 00:51:00.945082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:00.945038    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2" podStartSLOduration=1.945022622 podStartE2EDuration="1.945022622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:50:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:51:00.944381186 +0000 UTC m=+2810.668696371" watchObservedRunningTime="2026-07-23 00:51:00.945022622 +0000 UTC m=+2810.669337805"
Jul 23 00:51:01.299334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.299248    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4" podUID="358394b7-2288-4fab-9cf4-7ab9a761bc45" containerName="main" containerID="cri-o://ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73" gracePeriod=45
Jul 23 00:51:01.556514 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.556461    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:51:01.605427 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605401    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-dshm\") pod \"358394b7-2288-4fab-9cf4-7ab9a761bc45\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") "
Jul 23 00:51:01.605535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605457    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358394b7-2288-4fab-9cf4-7ab9a761bc45-tls-certs\") pod \"358394b7-2288-4fab-9cf4-7ab9a761bc45\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") "
Jul 23 00:51:01.605535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605487    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-model-cache\") pod \"358394b7-2288-4fab-9cf4-7ab9a761bc45\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") "
Jul 23 00:51:01.605608 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605551    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-tmp-dir\") pod \"358394b7-2288-4fab-9cf4-7ab9a761bc45\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") "
Jul 23 00:51:01.605653 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605610    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-home\") pod \"358394b7-2288-4fab-9cf4-7ab9a761bc45\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") "
Jul 23 00:51:01.605653 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605639    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dzvzk\" (UniqueName: \"kubernetes.io/projected/358394b7-2288-4fab-9cf4-7ab9a761bc45-kube-api-access-dzvzk\") pod \"358394b7-2288-4fab-9cf4-7ab9a761bc45\" (UID: \"358394b7-2288-4fab-9cf4-7ab9a761bc45\") "
Jul 23 00:51:01.605862 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605816    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-home" (OuterVolumeSpecName: "home") pod "358394b7-2288-4fab-9cf4-7ab9a761bc45" (UID: "358394b7-2288-4fab-9cf4-7ab9a761bc45"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:01.605862 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605847    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "358394b7-2288-4fab-9cf4-7ab9a761bc45" (UID: "358394b7-2288-4fab-9cf4-7ab9a761bc45"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:01.606012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605853    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-model-cache" (OuterVolumeSpecName: "model-cache") pod "358394b7-2288-4fab-9cf4-7ab9a761bc45" (UID: "358394b7-2288-4fab-9cf4-7ab9a761bc45"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:01.606012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.605988    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:51:01.606012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.606003    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:51:01.606177 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.606014    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:51:01.607640 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.607619    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/358394b7-2288-4fab-9cf4-7ab9a761bc45-kube-api-access-dzvzk" (OuterVolumeSpecName: "kube-api-access-dzvzk") pod "358394b7-2288-4fab-9cf4-7ab9a761bc45" (UID: "358394b7-2288-4fab-9cf4-7ab9a761bc45"). InnerVolumeSpecName "kube-api-access-dzvzk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:51:01.607728 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.607660    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-dshm" (OuterVolumeSpecName: "dshm") pod "358394b7-2288-4fab-9cf4-7ab9a761bc45" (UID: "358394b7-2288-4fab-9cf4-7ab9a761bc45"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:51:01.607728 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.607707    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/358394b7-2288-4fab-9cf4-7ab9a761bc45-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "358394b7-2288-4fab-9cf4-7ab9a761bc45" (UID: "358394b7-2288-4fab-9cf4-7ab9a761bc45"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:51:01.706511 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.706490    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/358394b7-2288-4fab-9cf4-7ab9a761bc45-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:51:01.706511 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.706512    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dzvzk\" (UniqueName: \"kubernetes.io/projected/358394b7-2288-4fab-9cf4-7ab9a761bc45-kube-api-access-dzvzk\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:51:01.706644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.706522    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/358394b7-2288-4fab-9cf4-7ab9a761bc45-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:51:01.931052 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.931022    2558 generic.go:356] "Generic (PLEG): container finished" podID="358394b7-2288-4fab-9cf4-7ab9a761bc45" containerID="ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73" exitCode=0
Jul 23 00:51:01.931395 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.931069    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"
Jul 23 00:51:01.931395 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.931102    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4" event={"ID":"358394b7-2288-4fab-9cf4-7ab9a761bc45","Type":"ContainerDied","Data":"ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73"}
Jul 23 00:51:01.931395 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.931142    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4" event={"ID":"358394b7-2288-4fab-9cf4-7ab9a761bc45","Type":"ContainerDied","Data":"30004e9af469f1a2257741ea8c559e509eed6cb59b3d17fc6cbe6440d7782fe9"}
Jul 23 00:51:01.931395 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.931159    2558 scope.go:117] "RemoveContainer" containerID="ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73"
Jul 23 00:51:01.940947 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.940930    2558 scope.go:117] "RemoveContainer" containerID="ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73"
Jul 23 00:51:01.941183 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:51:01.941165    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73\": container with ID starting with ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73 not found: ID does not exist" containerID="ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73"
Jul 23 00:51:01.941236 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.941190    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73"} err="failed to get container status \"ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73\": rpc error: code = NotFound desc = could not find container \"ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73\": container with ID starting with ac80eb2e91c408926faa8e924cafd2bcce02adaecf12ea37c53e2e821a86fa73 not found: ID does not exist"
Jul 23 00:51:01.952329 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.952306    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"]
Jul 23 00:51:01.958849 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:01.958821    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/auth-test-model-d50941-kserve-6777974947-2d7b4"]
Jul 23 00:51:02.735803 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:02.735770    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="358394b7-2288-4fab-9cf4-7ab9a761bc45" path="/var/lib/kubelet/pods/358394b7-2288-4fab-9cf4-7ab9a761bc45/volumes"
Jul 23 00:51:09.742732 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:09.742701    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:51:09.743106 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:09.742749    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:51:09.755273 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:09.755250    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:51:09.988462 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:09.988427    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:51:11.116319 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:11.116287    2558 scope.go:117] "RemoveContainer" containerID="0bd8b7c3238e075114aea4eaac9b1403a95362a1e7d3842dc1a809d5eabce376"
Jul 23 00:51:40.255553 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.255518    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"]
Jul 23 00:51:40.256075 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.255944    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="358394b7-2288-4fab-9cf4-7ab9a761bc45" containerName="main"
Jul 23 00:51:40.256075 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.255957    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="358394b7-2288-4fab-9cf4-7ab9a761bc45" containerName="main"
Jul 23 00:51:40.256075 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.256033    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="358394b7-2288-4fab-9cf4-7ab9a761bc45" containerName="main"
Jul 23 00:51:40.259182 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.259164    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.261594 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.261576    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"openshift-service-ca.crt\""
Jul 23 00:51:40.262331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.262317    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"default-dockercfg-69j6c\""
Jul 23 00:51:40.262438 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.262419    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"sub-test-model-d50941-kserve-self-signed-certs\""
Jul 23 00:51:40.262486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.262430    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"kube-root-ca.crt\""
Jul 23 00:51:40.268486 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.268460    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"]
Jul 23 00:51:40.326244 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.326216    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-dshm\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.326354 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.326337    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wwwc\" (UniqueName: \"kubernetes.io/projected/894c572a-5ead-4f36-9722-0bc17f8fee25-kube-api-access-7wwwc\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.326421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.326377    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-tmp-dir\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.326421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.326414    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-home\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.326512 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.326445    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-model-cache\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.326512 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.326482    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/894c572a-5ead-4f36-9722-0bc17f8fee25-tls-certs\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.426936 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.426905    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/894c572a-5ead-4f36-9722-0bc17f8fee25-tls-certs\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427056 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.426943    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-dshm\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427056 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427040    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwwc\" (UniqueName: \"kubernetes.io/projected/894c572a-5ead-4f36-9722-0bc17f8fee25-kube-api-access-7wwwc\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427129 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427074    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-tmp-dir\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427129 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427116    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-home\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427233 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427136    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-model-cache\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427470 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427448    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-tmp-dir\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427489    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-home\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.427552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.427532    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-model-cache\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.429305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.429289    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-dshm\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.429483 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.429466    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/894c572a-5ead-4f36-9722-0bc17f8fee25-tls-certs\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.445230 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.445203    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wwwc\" (UniqueName: \"kubernetes.io/projected/894c572a-5ead-4f36-9722-0bc17f8fee25-kube-api-access-7wwwc\") pod \"sub-test-model-d50941-kserve-789dbccd45-7x9nk\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") " pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.569252 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.569231    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:40.721757 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:40.721731    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"]
Jul 23 00:51:40.723162 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:51:40.723138    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod894c572a_5ead_4f36_9722_0bc17f8fee25.slice/crio-2b64be483e8de4e415e92a77f724343aaf80b5738081c73b36daa84e27d797d0 WatchSource:0}: Error finding container 2b64be483e8de4e415e92a77f724343aaf80b5738081c73b36daa84e27d797d0: Status 404 returned error can't find the container with id 2b64be483e8de4e415e92a77f724343aaf80b5738081c73b36daa84e27d797d0
Jul 23 00:51:41.156614 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:41.156570    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk" event={"ID":"894c572a-5ead-4f36-9722-0bc17f8fee25","Type":"ContainerStarted","Data":"dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c"}
Jul 23 00:51:41.156614 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:41.156620    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk" event={"ID":"894c572a-5ead-4f36-9722-0bc17f8fee25","Type":"ContainerStarted","Data":"2b64be483e8de4e415e92a77f724343aaf80b5738081c73b36daa84e27d797d0"}
Jul 23 00:51:50.570321 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:50.570283    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:50.570900 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:50.570340    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:50.583226 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:50.583201    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:51:50.604808 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:50.604758    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk" podStartSLOduration=10.604745687 podStartE2EDuration="10.604745687s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:51:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:51:41.206110827 +0000 UTC m=+2850.930426010" watchObservedRunningTime="2026-07-23 00:51:50.604745687 +0000 UTC m=+2860.329060914"
Jul 23 00:51:51.225472 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:51:51.225444    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:52:26.175961 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:26.175929    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"]
Jul 23 00:52:26.961366 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:26.961334    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"]
Jul 23 00:52:41.197302 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.197266    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2" podUID="68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" containerName="main" containerID="cri-o://fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727" gracePeriod=45
Jul 23 00:52:41.447067 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.447043    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:52:41.501276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.501204    2558 generic.go:356] "Generic (PLEG): container finished" podID="68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" containerID="fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727" exitCode=0
Jul 23 00:52:41.501276 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.501245    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"
Jul 23 00:52:41.501416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.501273    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2" event={"ID":"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97","Type":"ContainerDied","Data":"fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727"}
Jul 23 00:52:41.501416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.501305    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2" event={"ID":"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97","Type":"ContainerDied","Data":"ee164248b58aee03738b5e2bd24091923aa65d811dd559873406ebe31f52fc27"}
Jul 23 00:52:41.501416 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.501320    2558 scope.go:117] "RemoveContainer" containerID="fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727"
Jul 23 00:52:41.510303 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.510284    2558 scope.go:117] "RemoveContainer" containerID="fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727"
Jul 23 00:52:41.510561 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:52:41.510538    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727\": container with ID starting with fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727 not found: ID does not exist" containerID="fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727"
Jul 23 00:52:41.510612 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.510574    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727"} err="failed to get container status \"fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727\": rpc error: code = NotFound desc = could not find container \"fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727\": container with ID starting with fd7df369474b8d5296509ad25d8498b93a67ded9713b8cab6f3c293b91471727 not found: ID does not exist"
Jul 23 00:52:41.630379 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630356    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tmp-dir\") pod \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") "
Jul 23 00:52:41.630519 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630401    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-dshm\") pod \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") "
Jul 23 00:52:41.630519 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630427    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7pjgx\" (UniqueName: \"kubernetes.io/projected/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-kube-api-access-7pjgx\") pod \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") "
Jul 23 00:52:41.630519 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630474    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tls-certs\") pod \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") "
Jul 23 00:52:41.630692 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630577    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-model-cache\") pod \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") "
Jul 23 00:52:41.630692 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630606    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-home\") pod \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\" (UID: \"68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97\") "
Jul 23 00:52:41.630692 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630654    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" (UID: "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:41.630934 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630901    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-model-cache" (OuterVolumeSpecName: "model-cache") pod "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" (UID: "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:41.631014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630912    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:41.631014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.630940    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-home" (OuterVolumeSpecName: "home") pod "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" (UID: "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:41.632683 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.632658    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-dshm" (OuterVolumeSpecName: "dshm") pod "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" (UID: "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:41.632763 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.632726    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" (UID: "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:52:41.632802 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.632756    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-kube-api-access-7pjgx" (OuterVolumeSpecName: "kube-api-access-7pjgx") pod "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" (UID: "68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97"). InnerVolumeSpecName "kube-api-access-7pjgx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:52:41.732417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.732398    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:41.732552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.732529    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:41.732552 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.732552    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:41.732735 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.732564    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:41.732735 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.732577    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7pjgx\" (UniqueName: \"kubernetes.io/projected/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97-kube-api-access-7pjgx\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:41.826533 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.826508    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"]
Jul 23 00:52:41.828856 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.828823    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-59a77d/sub-test-model-59a77d-kserve-66469d54f7-psnz2"]
Jul 23 00:52:41.981947 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:41.981908    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk" podUID="894c572a-5ead-4f36-9722-0bc17f8fee25" containerName="main" containerID="cri-o://dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c" gracePeriod=45
Jul 23 00:52:42.241263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.241239    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:52:42.337587 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337561    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/894c572a-5ead-4f36-9722-0bc17f8fee25-tls-certs\") pod \"894c572a-5ead-4f36-9722-0bc17f8fee25\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") "
Jul 23 00:52:42.337720 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337600    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-tmp-dir\") pod \"894c572a-5ead-4f36-9722-0bc17f8fee25\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") "
Jul 23 00:52:42.337720 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337644    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-model-cache\") pod \"894c572a-5ead-4f36-9722-0bc17f8fee25\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") "
Jul 23 00:52:42.337720 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337686    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-dshm\") pod \"894c572a-5ead-4f36-9722-0bc17f8fee25\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") "
Jul 23 00:52:42.337720 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337714    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-home\") pod \"894c572a-5ead-4f36-9722-0bc17f8fee25\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") "
Jul 23 00:52:42.337977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337737    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wwwc\" (UniqueName: \"kubernetes.io/projected/894c572a-5ead-4f36-9722-0bc17f8fee25-kube-api-access-7wwwc\") pod \"894c572a-5ead-4f36-9722-0bc17f8fee25\" (UID: \"894c572a-5ead-4f36-9722-0bc17f8fee25\") "
Jul 23 00:52:42.337977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337928    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-home" (OuterVolumeSpecName: "home") pod "894c572a-5ead-4f36-9722-0bc17f8fee25" (UID: "894c572a-5ead-4f36-9722-0bc17f8fee25"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:42.337977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337949    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-model-cache" (OuterVolumeSpecName: "model-cache") pod "894c572a-5ead-4f36-9722-0bc17f8fee25" (UID: "894c572a-5ead-4f36-9722-0bc17f8fee25"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:42.337977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.337965    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "894c572a-5ead-4f36-9722-0bc17f8fee25" (UID: "894c572a-5ead-4f36-9722-0bc17f8fee25"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:42.338123 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.338020    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:42.338123 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.338038    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:42.339792 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.339768    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/894c572a-5ead-4f36-9722-0bc17f8fee25-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "894c572a-5ead-4f36-9722-0bc17f8fee25" (UID: "894c572a-5ead-4f36-9722-0bc17f8fee25"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:52:42.339891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.339770    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-dshm" (OuterVolumeSpecName: "dshm") pod "894c572a-5ead-4f36-9722-0bc17f8fee25" (UID: "894c572a-5ead-4f36-9722-0bc17f8fee25"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:52:42.339891 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.339874    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894c572a-5ead-4f36-9722-0bc17f8fee25-kube-api-access-7wwwc" (OuterVolumeSpecName: "kube-api-access-7wwwc") pod "894c572a-5ead-4f36-9722-0bc17f8fee25" (UID: "894c572a-5ead-4f36-9722-0bc17f8fee25"). InnerVolumeSpecName "kube-api-access-7wwwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:52:42.439371 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.439307    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:42.439371 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.439332    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wwwc\" (UniqueName: \"kubernetes.io/projected/894c572a-5ead-4f36-9722-0bc17f8fee25-kube-api-access-7wwwc\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:42.439371 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.439342    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/894c572a-5ead-4f36-9722-0bc17f8fee25-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:42.439371 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.439351    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/894c572a-5ead-4f36-9722-0bc17f8fee25-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:52:42.508572 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.508536    2558 generic.go:356] "Generic (PLEG): container finished" podID="894c572a-5ead-4f36-9722-0bc17f8fee25" containerID="dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c" exitCode=0
Jul 23 00:52:42.508729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.508583    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"
Jul 23 00:52:42.508729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.508618    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk" event={"ID":"894c572a-5ead-4f36-9722-0bc17f8fee25","Type":"ContainerDied","Data":"dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c"}
Jul 23 00:52:42.508729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.508666    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk" event={"ID":"894c572a-5ead-4f36-9722-0bc17f8fee25","Type":"ContainerDied","Data":"2b64be483e8de4e415e92a77f724343aaf80b5738081c73b36daa84e27d797d0"}
Jul 23 00:52:42.508729 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.508689    2558 scope.go:117] "RemoveContainer" containerID="dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c"
Jul 23 00:52:42.518059 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.518043    2558 scope.go:117] "RemoveContainer" containerID="dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c"
Jul 23 00:52:42.518301 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:52:42.518286    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c\": container with ID starting with dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c not found: ID does not exist" containerID="dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c"
Jul 23 00:52:42.518346 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.518308    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c"} err="failed to get container status \"dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c\": rpc error: code = NotFound desc = could not find container \"dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c\": container with ID starting with dbcbf54beb20661fd2f8b9ec756b72e1fd071aa0324efe89bf0bb5947e22ce9c not found: ID does not exist"
Jul 23 00:52:42.537228 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.537207    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"]
Jul 23 00:52:42.546238 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.546218    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/sub-test-model-d50941-kserve-789dbccd45-7x9nk"]
Jul 23 00:52:42.735434 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.735358    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" path="/var/lib/kubelet/pods/68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97/volumes"
Jul 23 00:52:42.735825 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:52:42.735806    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="894c572a-5ead-4f36-9722-0bc17f8fee25" path="/var/lib/kubelet/pods/894c572a-5ead-4f36-9722-0bc17f8fee25/volumes"
Jul 23 00:53:10.058173 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058137    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"]
Jul 23 00:53:10.058663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058548    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" containerName="main"
Jul 23 00:53:10.058663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058560    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" containerName="main"
Jul 23 00:53:10.058663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058576    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894c572a-5ead-4f36-9722-0bc17f8fee25" containerName="main"
Jul 23 00:53:10.058663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058581    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="894c572a-5ead-4f36-9722-0bc17f8fee25" containerName="main"
Jul 23 00:53:10.058663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058643    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="894c572a-5ead-4f36-9722-0bc17f8fee25" containerName="main"
Jul 23 00:53:10.058663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.058656    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="68ebd4be-bd96-4d2a-b4c3-44ba43f4eb97" containerName="main"
Jul 23 00:53:10.061851 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.061824    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.064602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.064575    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"openshift-service-ca.crt\""
Jul 23 00:53:10.065535 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.065518    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"kube-root-ca.crt\""
Jul 23 00:53:10.065648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.065520    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"rate-test-model-d50941-kserve-self-signed-certs\""
Jul 23 00:53:10.065770 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.065752    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-d50941\"/\"default-dockercfg-69j6c\""
Jul 23 00:53:10.071996 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.071975    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"]
Jul 23 00:53:10.166084 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.166056    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-dshm\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.166200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.166120    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-model-cache\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.166200 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.166154    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-tmp-dir\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.166323 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.166258    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jpcs\" (UniqueName: \"kubernetes.io/projected/06cf0266-a499-46da-8f27-fb517871d8bf-kube-api-access-9jpcs\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.166323 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.166304    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06cf0266-a499-46da-8f27-fb517871d8bf-tls-certs\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.166435 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.166329    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-home\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.266712 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.266684    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-model-cache\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.266825 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.266724    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-tmp-dir\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.266825 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.266800    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jpcs\" (UniqueName: \"kubernetes.io/projected/06cf0266-a499-46da-8f27-fb517871d8bf-kube-api-access-9jpcs\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.266986 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.266865    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06cf0266-a499-46da-8f27-fb517871d8bf-tls-certs\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.266986 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.266892    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-home\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.266986 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.266931    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-dshm\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.267148 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.267127    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-model-cache\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.267216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.267149    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-tmp-dir\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.267277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.267231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-home\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.269183 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.269161    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-dshm\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.269350 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.269332    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06cf0266-a499-46da-8f27-fb517871d8bf-tls-certs\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.277255 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.277231    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jpcs\" (UniqueName: \"kubernetes.io/projected/06cf0266-a499-46da-8f27-fb517871d8bf-kube-api-access-9jpcs\") pod \"rate-test-model-d50941-kserve-747fdfc7f-48zh9\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") " pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.371948 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.371872    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:10.513495 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.513466    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"]
Jul 23 00:53:10.514593 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:53:10.514568    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-f52fdc9bfd109f38ca346f7129c9fe918e8a2d30c1353096aab60c6bdb0af2b7 WatchSource:0}: Error finding container f52fdc9bfd109f38ca346f7129c9fe918e8a2d30c1353096aab60c6bdb0af2b7: Status 404 returned error can't find the container with id f52fdc9bfd109f38ca346f7129c9fe918e8a2d30c1353096aab60c6bdb0af2b7
Jul 23 00:53:10.678321 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.678233    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" event={"ID":"06cf0266-a499-46da-8f27-fb517871d8bf","Type":"ContainerStarted","Data":"41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be"}
Jul 23 00:53:10.678321 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.678278    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" event={"ID":"06cf0266-a499-46da-8f27-fb517871d8bf","Type":"ContainerStarted","Data":"f52fdc9bfd109f38ca346f7129c9fe918e8a2d30c1353096aab60c6bdb0af2b7"}
Jul 23 00:53:10.697767 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:10.697720    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" podStartSLOduration=0.697703714 podStartE2EDuration="697.703714ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:53:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:53:10.6956449 +0000 UTC m=+2940.419960100" watchObservedRunningTime="2026-07-23 00:53:10.697703714 +0000 UTC m=+2940.422018898"
Jul 23 00:53:20.372796 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:20.372755    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:20.373283 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:20.372805    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:20.385645 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:20.385616    2558 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:20.747387 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:20.747308    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:53:55.593363 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:55.593333    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"]
Jul 23 00:53:56.687439 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.687408    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-139842432982320"]
Jul 23 00:53:56.690959 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.690941    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:53:56.693064 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.693046    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-ptqjc\""
Jul 23 00:53:56.697392 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.697366    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842432982320"]
Jul 23 00:53:56.762344 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.762320    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df4vc\" (UniqueName: \"kubernetes.io/projected/b94ad75b-921a-4ac8-8015-e18cf338ee16-kube-api-access-df4vc\") pod \"test-curl-7694-139842432982320\" (UID: \"b94ad75b-921a-4ac8-8015-e18cf338ee16\") " pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:53:56.863687 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.863666    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df4vc\" (UniqueName: \"kubernetes.io/projected/b94ad75b-921a-4ac8-8015-e18cf338ee16-kube-api-access-df4vc\") pod \"test-curl-7694-139842432982320\" (UID: \"b94ad75b-921a-4ac8-8015-e18cf338ee16\") " pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:53:56.871963 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:56.871934    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df4vc\" (UniqueName: \"kubernetes.io/projected/b94ad75b-921a-4ac8-8015-e18cf338ee16-kube-api-access-df4vc\") pod \"test-curl-7694-139842432982320\" (UID: \"b94ad75b-921a-4ac8-8015-e18cf338ee16\") " pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:53:57.002359 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:57.002296    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:53:57.339144 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:57.339118    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842432982320"]
Jul 23 00:53:57.340769 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:53:57.340741    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb94ad75b_921a_4ac8_8015_e18cf338ee16.slice/crio-4ae626c9555bbbf7a2c22464353456c204573b2119da6610bffd94d631c72f9c WatchSource:0}: Error finding container 4ae626c9555bbbf7a2c22464353456c204573b2119da6610bffd94d631c72f9c: Status 404 returned error can't find the container with id 4ae626c9555bbbf7a2c22464353456c204573b2119da6610bffd94d631c72f9c
Jul 23 00:53:57.342231 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:57.342214    2558 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 23 00:53:57.944057 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:57.944018    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842432982320" event={"ID":"b94ad75b-921a-4ac8-8015-e18cf338ee16","Type":"ContainerStarted","Data":"4ae626c9555bbbf7a2c22464353456c204573b2119da6610bffd94d631c72f9c"}
Jul 23 00:53:58.950880 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:58.950850    2558 generic.go:356] "Generic (PLEG): container finished" podID="b94ad75b-921a-4ac8-8015-e18cf338ee16" containerID="fdb8198f1837cb453a65e38493978d77c379e506ccdc2bed4c701b77ca00b417" exitCode=0
Jul 23 00:53:58.951243 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:53:58.950927    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842432982320" event={"ID":"b94ad75b-921a-4ac8-8015-e18cf338ee16","Type":"ContainerDied","Data":"fdb8198f1837cb453a65e38493978d77c379e506ccdc2bed4c701b77ca00b417"}
Jul 23 00:54:00.106780 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.106755    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:54:00.141768 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.141744    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-139842432982320_b94ad75b-921a-4ac8-8015-e18cf338ee16/test-curl-7694-139842432982320/0.log"
Jul 23 00:54:00.167274 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.167249    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-139842432982320"]
Jul 23 00:54:00.171038 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.171017    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-139842432982320"]
Jul 23 00:54:00.196875 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.196829    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-df4vc\" (UniqueName: \"kubernetes.io/projected/b94ad75b-921a-4ac8-8015-e18cf338ee16-kube-api-access-df4vc\") pod \"b94ad75b-921a-4ac8-8015-e18cf338ee16\" (UID: \"b94ad75b-921a-4ac8-8015-e18cf338ee16\") "
Jul 23 00:54:00.199257 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.199228    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b94ad75b-921a-4ac8-8015-e18cf338ee16-kube-api-access-df4vc" (OuterVolumeSpecName: "kube-api-access-df4vc") pod "b94ad75b-921a-4ac8-8015-e18cf338ee16" (UID: "b94ad75b-921a-4ac8-8015-e18cf338ee16"). InnerVolumeSpecName "kube-api-access-df4vc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:00.298295 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.298227    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-df4vc\" (UniqueName: \"kubernetes.io/projected/b94ad75b-921a-4ac8-8015-e18cf338ee16-kube-api-access-df4vc\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:00.308346 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.308316    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-139842432981552"]
Jul 23 00:54:00.308745 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.308731    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b94ad75b-921a-4ac8-8015-e18cf338ee16" containerName="test-curl-7694-139842432982320"
Jul 23 00:54:00.308796 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.308746    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="b94ad75b-921a-4ac8-8015-e18cf338ee16" containerName="test-curl-7694-139842432982320"
Jul 23 00:54:00.308844 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.308825    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="b94ad75b-921a-4ac8-8015-e18cf338ee16" containerName="test-curl-7694-139842432982320"
Jul 23 00:54:00.312104 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.312087    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:00.318830 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.318808    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842432981552"]
Jul 23 00:54:00.399417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.399392    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49jbl\" (UniqueName: \"kubernetes.io/projected/fe524cee-a820-4526-8574-8cab441d6030-kube-api-access-49jbl\") pod \"test-curl-7694-139842432981552\" (UID: \"fe524cee-a820-4526-8574-8cab441d6030\") " pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:00.500009 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.499988    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49jbl\" (UniqueName: \"kubernetes.io/projected/fe524cee-a820-4526-8574-8cab441d6030-kube-api-access-49jbl\") pod \"test-curl-7694-139842432981552\" (UID: \"fe524cee-a820-4526-8574-8cab441d6030\") " pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:00.508362 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.508332    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49jbl\" (UniqueName: \"kubernetes.io/projected/fe524cee-a820-4526-8574-8cab441d6030-kube-api-access-49jbl\") pod \"test-curl-7694-139842432981552\" (UID: \"fe524cee-a820-4526-8574-8cab441d6030\") " pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:00.622401 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.622365    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:00.737490 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.737457    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b94ad75b-921a-4ac8-8015-e18cf338ee16" path="/var/lib/kubelet/pods/b94ad75b-921a-4ac8-8015-e18cf338ee16/volumes"
Jul 23 00:54:00.963234 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.963154    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432982320"
Jul 23 00:54:00.963234 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.963196    2558 scope.go:117] "RemoveContainer" containerID="fdb8198f1837cb453a65e38493978d77c379e506ccdc2bed4c701b77ca00b417"
Jul 23 00:54:00.965212 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:00.965133    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842432981552"]
Jul 23 00:54:00.966434 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:54:00.966414    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfe524cee_a820_4526_8574_8cab441d6030.slice/crio-2e3a1d1c594aa84b1e3fe44e1cd23cfba2a276ebf2936ae4cb9999945a11ac12 WatchSource:0}: Error finding container 2e3a1d1c594aa84b1e3fe44e1cd23cfba2a276ebf2936ae4cb9999945a11ac12: Status 404 returned error can't find the container with id 2e3a1d1c594aa84b1e3fe44e1cd23cfba2a276ebf2936ae4cb9999945a11ac12
Jul 23 00:54:01.970723 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:01.970690    2558 generic.go:356] "Generic (PLEG): container finished" podID="fe524cee-a820-4526-8574-8cab441d6030" containerID="51388324c84f328d8a382def28b632052e32e370b64cf562938ff98e9fbfee40" exitCode=0
Jul 23 00:54:01.971140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:01.970769    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842432981552" event={"ID":"fe524cee-a820-4526-8574-8cab441d6030","Type":"ContainerDied","Data":"51388324c84f328d8a382def28b632052e32e370b64cf562938ff98e9fbfee40"}
Jul 23 00:54:01.971140 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:01.970793    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842432981552" event={"ID":"fe524cee-a820-4526-8574-8cab441d6030","Type":"ContainerStarted","Data":"2e3a1d1c594aa84b1e3fe44e1cd23cfba2a276ebf2936ae4cb9999945a11ac12"}
Jul 23 00:54:03.109205 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.109178    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:03.128717 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.128695    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-139842432981552_fe524cee-a820-4526-8574-8cab441d6030/test-curl-7694-139842432981552/0.log"
Jul 23 00:54:03.150965 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.150936    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-139842432981552"]
Jul 23 00:54:03.153207 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.153189    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-139842432981552"]
Jul 23 00:54:03.222702 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.222679    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49jbl\" (UniqueName: \"kubernetes.io/projected/fe524cee-a820-4526-8574-8cab441d6030-kube-api-access-49jbl\") pod \"fe524cee-a820-4526-8574-8cab441d6030\" (UID: \"fe524cee-a820-4526-8574-8cab441d6030\") "
Jul 23 00:54:03.224611 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.224590    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe524cee-a820-4526-8574-8cab441d6030-kube-api-access-49jbl" (OuterVolumeSpecName: "kube-api-access-49jbl") pod "fe524cee-a820-4526-8574-8cab441d6030" (UID: "fe524cee-a820-4526-8574-8cab441d6030"). InnerVolumeSpecName "kube-api-access-49jbl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:03.324001 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.323979    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49jbl\" (UniqueName: \"kubernetes.io/projected/fe524cee-a820-4526-8574-8cab441d6030-kube-api-access-49jbl\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:03.324305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.324287    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:03.324719 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.324704    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe524cee-a820-4526-8574-8cab441d6030" containerName="test-curl-7694-139842432981552"
Jul 23 00:54:03.324760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.324721    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe524cee-a820-4526-8574-8cab441d6030" containerName="test-curl-7694-139842432981552"
Jul 23 00:54:03.324798 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.324789    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe524cee-a820-4526-8574-8cab441d6030" containerName="test-curl-7694-139842432981552"
Jul 23 00:54:03.327711 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.327696    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:03.335956 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.335934    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:03.424385 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.424354    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lt9m8\" (UniqueName: \"kubernetes.io/projected/84611cc8-2402-40b8-80e6-b41f247e9db5-kube-api-access-lt9m8\") pod \"test-curl-7694-139842433053728\" (UID: \"84611cc8-2402-40b8-80e6-b41f247e9db5\") " pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:03.525447 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.525378    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lt9m8\" (UniqueName: \"kubernetes.io/projected/84611cc8-2402-40b8-80e6-b41f247e9db5-kube-api-access-lt9m8\") pod \"test-curl-7694-139842433053728\" (UID: \"84611cc8-2402-40b8-80e6-b41f247e9db5\") " pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:03.534009 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.533988    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lt9m8\" (UniqueName: \"kubernetes.io/projected/84611cc8-2402-40b8-80e6-b41f247e9db5-kube-api-access-lt9m8\") pod \"test-curl-7694-139842433053728\" (UID: \"84611cc8-2402-40b8-80e6-b41f247e9db5\") " pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:03.637415 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.637389    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:03.769606 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.769576    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:03.771412 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:54:03.771382    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod84611cc8_2402_40b8_80e6_b41f247e9db5.slice/crio-e6ea436935ef4528ef9d8eebd3ca7a290a09e3f49a07371f9daed13900097792 WatchSource:0}: Error finding container e6ea436935ef4528ef9d8eebd3ca7a290a09e3f49a07371f9daed13900097792: Status 404 returned error can't find the container with id e6ea436935ef4528ef9d8eebd3ca7a290a09e3f49a07371f9daed13900097792
Jul 23 00:54:03.983956 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.983931    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:03.983956 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.983953    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2e3a1d1c594aa84b1e3fe44e1cd23cfba2a276ebf2936ae4cb9999945a11ac12"
Jul 23 00:54:03.985205 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:03.985179    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433053728" event={"ID":"84611cc8-2402-40b8-80e6-b41f247e9db5","Type":"ContainerStarted","Data":"e6ea436935ef4528ef9d8eebd3ca7a290a09e3f49a07371f9daed13900097792"}
Jul 23 00:54:03.994275 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:03.994253    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-139842432981552\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="fe524cee-a820-4526-8574-8cab441d6030" pod="opendatahub/test-curl-7694-139842432981552"
Jul 23 00:54:04.735353 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:04.735316    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe524cee-a820-4526-8574-8cab441d6030" path="/var/lib/kubelet/pods/fe524cee-a820-4526-8574-8cab441d6030/volumes"
Jul 23 00:54:04.992780 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:04.992692    2558 generic.go:356] "Generic (PLEG): container finished" podID="84611cc8-2402-40b8-80e6-b41f247e9db5" containerID="6fd1a69b76dc3bd3c2ba928d43e56b3f48a3118a16c27080fd3cc49b484adfb9" exitCode=0
Jul 23 00:54:04.992780 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:04.992743    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433053728" event={"ID":"84611cc8-2402-40b8-80e6-b41f247e9db5","Type":"ContainerDied","Data":"6fd1a69b76dc3bd3c2ba928d43e56b3f48a3118a16c27080fd3cc49b484adfb9"}
Jul 23 00:54:06.134592 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.134568    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:06.154240 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.154216    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-139842433053728_84611cc8-2402-40b8-80e6-b41f247e9db5/test-curl-7694-139842433053728/0.log"
Jul 23 00:54:06.178488 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.178458    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:06.180805 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.180782    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:06.250191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.250167    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lt9m8\" (UniqueName: \"kubernetes.io/projected/84611cc8-2402-40b8-80e6-b41f247e9db5-kube-api-access-lt9m8\") pod \"84611cc8-2402-40b8-80e6-b41f247e9db5\" (UID: \"84611cc8-2402-40b8-80e6-b41f247e9db5\") "
Jul 23 00:54:06.252240 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.252185    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84611cc8-2402-40b8-80e6-b41f247e9db5-kube-api-access-lt9m8" (OuterVolumeSpecName: "kube-api-access-lt9m8") pod "84611cc8-2402-40b8-80e6-b41f247e9db5" (UID: "84611cc8-2402-40b8-80e6-b41f247e9db5"). InnerVolumeSpecName "kube-api-access-lt9m8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:06.310188 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.310160    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:06.310635 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.310618    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84611cc8-2402-40b8-80e6-b41f247e9db5" containerName="test-curl-7694-139842433053728"
Jul 23 00:54:06.310722 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.310637    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="84611cc8-2402-40b8-80e6-b41f247e9db5" containerName="test-curl-7694-139842433053728"
Jul 23 00:54:06.310786 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.310757    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="84611cc8-2402-40b8-80e6-b41f247e9db5" containerName="test-curl-7694-139842433053728"
Jul 23 00:54:06.313900 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.313882    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:06.319437 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.319414    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:06.350916 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.350884    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lt9m8\" (UniqueName: \"kubernetes.io/projected/84611cc8-2402-40b8-80e6-b41f247e9db5-kube-api-access-lt9m8\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:06.452263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.452239    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhv6z\" (UniqueName: \"kubernetes.io/projected/9d35d62f-2978-418c-8ec4-08f07d279ee5-kube-api-access-vhv6z\") pod \"test-curl-7694-139842433053728\" (UID: \"9d35d62f-2978-418c-8ec4-08f07d279ee5\") " pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:06.553677 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.553618    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhv6z\" (UniqueName: \"kubernetes.io/projected/9d35d62f-2978-418c-8ec4-08f07d279ee5-kube-api-access-vhv6z\") pod \"test-curl-7694-139842433053728\" (UID: \"9d35d62f-2978-418c-8ec4-08f07d279ee5\") " pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:06.564229 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.564201    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhv6z\" (UniqueName: \"kubernetes.io/projected/9d35d62f-2978-418c-8ec4-08f07d279ee5-kube-api-access-vhv6z\") pod \"test-curl-7694-139842433053728\" (UID: \"9d35d62f-2978-418c-8ec4-08f07d279ee5\") " pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:06.624331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.624311    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:06.737355 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.737324    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84611cc8-2402-40b8-80e6-b41f247e9db5" path="/var/lib/kubelet/pods/84611cc8-2402-40b8-80e6-b41f247e9db5/volumes"
Jul 23 00:54:06.758343 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:06.758322    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:06.759501 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:54:06.759476    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d35d62f_2978_418c_8ec4_08f07d279ee5.slice/crio-5f894c458fab8ad422ec0a8786e6444718deb858a1f70e3b35e10bb5056d6485 WatchSource:0}: Error finding container 5f894c458fab8ad422ec0a8786e6444718deb858a1f70e3b35e10bb5056d6485: Status 404 returned error can't find the container with id 5f894c458fab8ad422ec0a8786e6444718deb858a1f70e3b35e10bb5056d6485
Jul 23 00:54:07.008127 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:07.008089    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433053728" event={"ID":"9d35d62f-2978-418c-8ec4-08f07d279ee5","Type":"ContainerStarted","Data":"5f894c458fab8ad422ec0a8786e6444718deb858a1f70e3b35e10bb5056d6485"}
Jul 23 00:54:07.009263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:07.009243    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:07.009369 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:07.009262    2558 scope.go:117] "RemoveContainer" containerID="6fd1a69b76dc3bd3c2ba928d43e56b3f48a3118a16c27080fd3cc49b484adfb9"
Jul 23 00:54:08.017489 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:08.017456    2558 generic.go:356] "Generic (PLEG): container finished" podID="9d35d62f-2978-418c-8ec4-08f07d279ee5" containerID="efa8569daa5432c50170f4c35a63ed5afdde20bb4c8cef1f3a67dfa485e8bbd5" exitCode=0
Jul 23 00:54:08.017903 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:08.017543    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433053728" event={"ID":"9d35d62f-2978-418c-8ec4-08f07d279ee5","Type":"ContainerDied","Data":"efa8569daa5432c50170f4c35a63ed5afdde20bb4c8cef1f3a67dfa485e8bbd5"}
Jul 23 00:54:09.161065 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.161045    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:09.180722 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.180623    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-139842433053728_9d35d62f-2978-418c-8ec4-08f07d279ee5/test-curl-7694-139842433053728/0.log"
Jul 23 00:54:09.204905 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.204874    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:09.207233 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.207209    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-139842433053728"]
Jul 23 00:54:09.278302 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.278227    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vhv6z\" (UniqueName: \"kubernetes.io/projected/9d35d62f-2978-418c-8ec4-08f07d279ee5-kube-api-access-vhv6z\") pod \"9d35d62f-2978-418c-8ec4-08f07d279ee5\" (UID: \"9d35d62f-2978-418c-8ec4-08f07d279ee5\") "
Jul 23 00:54:09.280248 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.280225    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d35d62f-2978-418c-8ec4-08f07d279ee5-kube-api-access-vhv6z" (OuterVolumeSpecName: "kube-api-access-vhv6z") pod "9d35d62f-2978-418c-8ec4-08f07d279ee5" (UID: "9d35d62f-2978-418c-8ec4-08f07d279ee5"). InnerVolumeSpecName "kube-api-access-vhv6z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:09.339002 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.338980    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:09.339382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.339370    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d35d62f-2978-418c-8ec4-08f07d279ee5" containerName="test-curl-7694-139842433053728"
Jul 23 00:54:09.339429 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.339384    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d35d62f-2978-418c-8ec4-08f07d279ee5" containerName="test-curl-7694-139842433053728"
Jul 23 00:54:09.339470 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.339451    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d35d62f-2978-418c-8ec4-08f07d279ee5" containerName="test-curl-7694-139842433053728"
Jul 23 00:54:09.342443 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.342422    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:09.346553 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.346533    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:09.379207 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.379183    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vhv6z\" (UniqueName: \"kubernetes.io/projected/9d35d62f-2978-418c-8ec4-08f07d279ee5-kube-api-access-vhv6z\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:09.480016 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.479992    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4cx4\" (UniqueName: \"kubernetes.io/projected/46172d00-e4d1-443a-b85c-be7c60049dcb-kube-api-access-z4cx4\") pod \"test-curl-7694-139842433054736\" (UID: \"46172d00-e4d1-443a-b85c-be7c60049dcb\") " pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:09.580565 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.580541    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cx4\" (UniqueName: \"kubernetes.io/projected/46172d00-e4d1-443a-b85c-be7c60049dcb-kube-api-access-z4cx4\") pod \"test-curl-7694-139842433054736\" (UID: \"46172d00-e4d1-443a-b85c-be7c60049dcb\") " pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:09.588798 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.588778    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4cx4\" (UniqueName: \"kubernetes.io/projected/46172d00-e4d1-443a-b85c-be7c60049dcb-kube-api-access-z4cx4\") pod \"test-curl-7694-139842433054736\" (UID: \"46172d00-e4d1-443a-b85c-be7c60049dcb\") " pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:09.652918 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.652898    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:09.989655 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:09.989603    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:10.031415 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.031371    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433054736" event={"ID":"46172d00-e4d1-443a-b85c-be7c60049dcb","Type":"ContainerStarted","Data":"65cd52f9aa5556172b9a00c79a0cea6d3b585ca4967218f3904944e1e06e84d8"}
Jul 23 00:54:10.032635 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.032616    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:10.032737 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.032616    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5f894c458fab8ad422ec0a8786e6444718deb858a1f70e3b35e10bb5056d6485"
Jul 23 00:54:10.043477 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:10.043446    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-139842433053728\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="9d35d62f-2978-418c-8ec4-08f07d279ee5" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:10.613534 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.613502    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" podUID="06cf0266-a499-46da-8f27-fb517871d8bf" containerName="main" containerID="cri-o://41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be" gracePeriod=45
Jul 23 00:54:10.734041 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:10.734008    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-139842433053728\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="9d35d62f-2978-418c-8ec4-08f07d279ee5" pod="opendatahub/test-curl-7694-139842433053728"
Jul 23 00:54:10.739396 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.739350    2558 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" podUID="06cf0266-a499-46da-8f27-fb517871d8bf" containerName="main" probeResult="failure" output="Get \"https://10.132.0.76:8000/ready\": dial tcp 10.132.0.76:8000: connect: connection refused"
Jul 23 00:54:10.740201 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.740177    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d35d62f-2978-418c-8ec4-08f07d279ee5" path="/var/lib/kubelet/pods/9d35d62f-2978-418c-8ec4-08f07d279ee5/volumes"
Jul 23 00:54:10.866997 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.866945    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:54:10.991571 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.991541    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:54:10.995495 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995470    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-home\") pod \"06cf0266-a499-46da-8f27-fb517871d8bf\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") "
Jul 23 00:54:10.995637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995520    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06cf0266-a499-46da-8f27-fb517871d8bf-tls-certs\") pod \"06cf0266-a499-46da-8f27-fb517871d8bf\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") "
Jul 23 00:54:10.995637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995550    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-dshm\") pod \"06cf0266-a499-46da-8f27-fb517871d8bf\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") "
Jul 23 00:54:10.995637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995580    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-tmp-dir\") pod \"06cf0266-a499-46da-8f27-fb517871d8bf\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") "
Jul 23 00:54:10.995813 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995665    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-model-cache\") pod \"06cf0266-a499-46da-8f27-fb517871d8bf\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") "
Jul 23 00:54:10.995813 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995708    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jpcs\" (UniqueName: \"kubernetes.io/projected/06cf0266-a499-46da-8f27-fb517871d8bf-kube-api-access-9jpcs\") pod \"06cf0266-a499-46da-8f27-fb517871d8bf\" (UID: \"06cf0266-a499-46da-8f27-fb517871d8bf\") "
Jul 23 00:54:10.995813 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995785    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-home" (OuterVolumeSpecName: "home") pod "06cf0266-a499-46da-8f27-fb517871d8bf" (UID: "06cf0266-a499-46da-8f27-fb517871d8bf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:54:10.996004 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995865    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "06cf0266-a499-46da-8f27-fb517871d8bf" (UID: "06cf0266-a499-46da-8f27-fb517871d8bf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:54:10.996060 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.995996    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-model-cache" (OuterVolumeSpecName: "model-cache") pod "06cf0266-a499-46da-8f27-fb517871d8bf" (UID: "06cf0266-a499-46da-8f27-fb517871d8bf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:54:10.996195 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.996176    2558 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-home\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:10.996265 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.996223    2558 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-tmp-dir\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:10.996265 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.996238    2558 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-model-cache\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:10.997986 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.997963    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06cf0266-a499-46da-8f27-fb517871d8bf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "06cf0266-a499-46da-8f27-fb517871d8bf" (UID: "06cf0266-a499-46da-8f27-fb517871d8bf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 23 00:54:10.998151 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.998125    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06cf0266-a499-46da-8f27-fb517871d8bf-kube-api-access-9jpcs" (OuterVolumeSpecName: "kube-api-access-9jpcs") pod "06cf0266-a499-46da-8f27-fb517871d8bf" (UID: "06cf0266-a499-46da-8f27-fb517871d8bf"). InnerVolumeSpecName "kube-api-access-9jpcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:10.998267 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:10.998190    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-dshm" (OuterVolumeSpecName: "dshm") pod "06cf0266-a499-46da-8f27-fb517871d8bf" (UID: "06cf0266-a499-46da-8f27-fb517871d8bf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:54:11.001405 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.001389    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-72k8d_75e5c474-9fc2-4a4d-bc80-8f4c5dddc47a/ovn-acl-logging/0.log"
Jul 23 00:54:11.039094 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.039067    2558 generic.go:356] "Generic (PLEG): container finished" podID="06cf0266-a499-46da-8f27-fb517871d8bf" containerID="41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be" exitCode=0
Jul 23 00:54:11.039185 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.039118    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"
Jul 23 00:54:11.039185 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.039153    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" event={"ID":"06cf0266-a499-46da-8f27-fb517871d8bf","Type":"ContainerDied","Data":"41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be"}
Jul 23 00:54:11.039307 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.039198    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9" event={"ID":"06cf0266-a499-46da-8f27-fb517871d8bf","Type":"ContainerDied","Data":"f52fdc9bfd109f38ca346f7129c9fe918e8a2d30c1353096aab60c6bdb0af2b7"}
Jul 23 00:54:11.039307 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.039220    2558 scope.go:117] "RemoveContainer" containerID="41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be"
Jul 23 00:54:11.040952 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.040928    2558 generic.go:356] "Generic (PLEG): container finished" podID="46172d00-e4d1-443a-b85c-be7c60049dcb" containerID="0bf1d953c941a810aa2ea2d0cb974a1d7e693e5c2983eea41bfbb8f250568e86" exitCode=0
Jul 23 00:54:11.041022 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.041007    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433054736" event={"ID":"46172d00-e4d1-443a-b85c-be7c60049dcb","Type":"ContainerDied","Data":"0bf1d953c941a810aa2ea2d0cb974a1d7e693e5c2983eea41bfbb8f250568e86"}
Jul 23 00:54:11.048216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.048198    2558 scope.go:117] "RemoveContainer" containerID="41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be"
Jul 23 00:54:11.048448 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:11.048430    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be\": container with ID starting with 41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be not found: ID does not exist" containerID="41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be"
Jul 23 00:54:11.048494 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.048456    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be"} err="failed to get container status \"41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be\": rpc error: code = NotFound desc = could not find container \"41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be\": container with ID starting with 41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be not found: ID does not exist"
Jul 23 00:54:11.090066 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.090042    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"]
Jul 23 00:54:11.092130 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.092111    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-d50941/rate-test-model-d50941-kserve-747fdfc7f-48zh9"]
Jul 23 00:54:11.097604 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.097586    2558 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06cf0266-a499-46da-8f27-fb517871d8bf-tls-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:11.097676 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.097607    2558 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06cf0266-a499-46da-8f27-fb517871d8bf-dshm\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:11.097676 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:11.097617    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jpcs\" (UniqueName: \"kubernetes.io/projected/06cf0266-a499-46da-8f27-fb517871d8bf-kube-api-access-9jpcs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:11.470496 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:11.470414    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:12.185905 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.185878    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:12.205810 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.205787    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-139842433054736_46172d00-e4d1-443a-b85c-be7c60049dcb/test-curl-7694-139842433054736/0.log"
Jul 23 00:54:12.227806 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.227782    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:12.233616 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.233594    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:12.308796 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.308774    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4cx4\" (UniqueName: \"kubernetes.io/projected/46172d00-e4d1-443a-b85c-be7c60049dcb-kube-api-access-z4cx4\") pod \"46172d00-e4d1-443a-b85c-be7c60049dcb\" (UID: \"46172d00-e4d1-443a-b85c-be7c60049dcb\") "
Jul 23 00:54:12.310817 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.310793    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46172d00-e4d1-443a-b85c-be7c60049dcb-kube-api-access-z4cx4" (OuterVolumeSpecName: "kube-api-access-z4cx4") pod "46172d00-e4d1-443a-b85c-be7c60049dcb" (UID: "46172d00-e4d1-443a-b85c-be7c60049dcb"). InnerVolumeSpecName "kube-api-access-z4cx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:12.372392 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372367    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:12.372810 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372797    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46172d00-e4d1-443a-b85c-be7c60049dcb" containerName="test-curl-7694-139842433054736"
Jul 23 00:54:12.372869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372812    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="46172d00-e4d1-443a-b85c-be7c60049dcb" containerName="test-curl-7694-139842433054736"
Jul 23 00:54:12.372869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372831    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06cf0266-a499-46da-8f27-fb517871d8bf" containerName="main"
Jul 23 00:54:12.372869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372850    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="06cf0266-a499-46da-8f27-fb517871d8bf" containerName="main"
Jul 23 00:54:12.372970 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372930    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="06cf0266-a499-46da-8f27-fb517871d8bf" containerName="main"
Jul 23 00:54:12.372970 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.372940    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="46172d00-e4d1-443a-b85c-be7c60049dcb" containerName="test-curl-7694-139842433054736"
Jul 23 00:54:12.376057 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.376041    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:12.380290 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.380267    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:12.410245 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.410219    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4cx4\" (UniqueName: \"kubernetes.io/projected/46172d00-e4d1-443a-b85c-be7c60049dcb-kube-api-access-z4cx4\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:12.510828 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.510800    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrqjw\" (UniqueName: \"kubernetes.io/projected/a29a8e36-8ad9-48c0-adea-650195f39943-kube-api-access-nrqjw\") pod \"test-curl-7694-139842433054736\" (UID: \"a29a8e36-8ad9-48c0-adea-650195f39943\") " pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:12.611317 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.611248    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrqjw\" (UniqueName: \"kubernetes.io/projected/a29a8e36-8ad9-48c0-adea-650195f39943-kube-api-access-nrqjw\") pod \"test-curl-7694-139842433054736\" (UID: \"a29a8e36-8ad9-48c0-adea-650195f39943\") " pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:12.621352 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.621325    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrqjw\" (UniqueName: \"kubernetes.io/projected/a29a8e36-8ad9-48c0-adea-650195f39943-kube-api-access-nrqjw\") pod \"test-curl-7694-139842433054736\" (UID: \"a29a8e36-8ad9-48c0-adea-650195f39943\") " pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:12.686644 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.686622    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:12.735253 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.735225    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06cf0266-a499-46da-8f27-fb517871d8bf" path="/var/lib/kubelet/pods/06cf0266-a499-46da-8f27-fb517871d8bf/volumes"
Jul 23 00:54:12.735946 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.735685    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46172d00-e4d1-443a-b85c-be7c60049dcb" path="/var/lib/kubelet/pods/46172d00-e4d1-443a-b85c-be7c60049dcb/volumes"
Jul 23 00:54:12.846648 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:12.846621    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:12.848368 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:54:12.848343    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda29a8e36_8ad9_48c0_adea_650195f39943.slice/crio-adf999308b2f2b12da4abcf44da6679bafad866e9a598040e769e8c29d0343da WatchSource:0}: Error finding container adf999308b2f2b12da4abcf44da6679bafad866e9a598040e769e8c29d0343da: Status 404 returned error can't find the container with id adf999308b2f2b12da4abcf44da6679bafad866e9a598040e769e8c29d0343da
Jul 23 00:54:13.053191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:13.053164    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433054736" event={"ID":"a29a8e36-8ad9-48c0-adea-650195f39943","Type":"ContainerStarted","Data":"adf999308b2f2b12da4abcf44da6679bafad866e9a598040e769e8c29d0343da"}
Jul 23 00:54:13.054382 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:13.054364    2558 scope.go:117] "RemoveContainer" containerID="0bf1d953c941a810aa2ea2d0cb974a1d7e693e5c2983eea41bfbb8f250568e86"
Jul 23 00:54:13.054491 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:13.054377    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:14.062610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:14.062579    2558 generic.go:356] "Generic (PLEG): container finished" podID="a29a8e36-8ad9-48c0-adea-650195f39943" containerID="67d9cd1396cd97b249cc9e494af6c1906cc6dbbaf665f1668fc1619960eaf084" exitCode=0
Jul 23 00:54:14.063016 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:14.062642    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7694-139842433054736" event={"ID":"a29a8e36-8ad9-48c0-adea-650195f39943","Type":"ContainerDied","Data":"67d9cd1396cd97b249cc9e494af6c1906cc6dbbaf665f1668fc1619960eaf084"}
Jul 23 00:54:14.652158 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:14.652119    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:14.652386 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:14.652361    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:14.694313 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:14.694284    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:15.222482 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.222457    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:15.243503 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.243479    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7694-139842433054736_a29a8e36-8ad9-48c0-adea-650195f39943/test-curl-7694-139842433054736/0.log"
Jul 23 00:54:15.268932 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.268900    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:15.272446 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.272424    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7694-139842433054736"]
Jul 23 00:54:15.337800 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.337744    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrqjw\" (UniqueName: \"kubernetes.io/projected/a29a8e36-8ad9-48c0-adea-650195f39943-kube-api-access-nrqjw\") pod \"a29a8e36-8ad9-48c0-adea-650195f39943\" (UID: \"a29a8e36-8ad9-48c0-adea-650195f39943\") "
Jul 23 00:54:15.339804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.339774    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a29a8e36-8ad9-48c0-adea-650195f39943-kube-api-access-nrqjw" (OuterVolumeSpecName: "kube-api-access-nrqjw") pod "a29a8e36-8ad9-48c0-adea-650195f39943" (UID: "a29a8e36-8ad9-48c0-adea-650195f39943"). InnerVolumeSpecName "kube-api-access-nrqjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:54:15.438645 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.438618    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrqjw\" (UniqueName: \"kubernetes.io/projected/a29a8e36-8ad9-48c0-adea-650195f39943-kube-api-access-nrqjw\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:15.481806 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.481781    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"]
Jul 23 00:54:15.482185 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.482172    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a29a8e36-8ad9-48c0-adea-650195f39943" containerName="test-curl-7694-139842433054736"
Jul 23 00:54:15.482283 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.482187    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="a29a8e36-8ad9-48c0-adea-650195f39943" containerName="test-curl-7694-139842433054736"
Jul 23 00:54:15.482283 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.482276    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="a29a8e36-8ad9-48c0-adea-650195f39943" containerName="test-curl-7694-139842433054736"
Jul 23 00:54:15.485305 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.485290    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:15.487421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.487402    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 23 00:54:15.487515 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.487475    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5dc6x\""
Jul 23 00:54:15.494277 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.493642    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"]
Jul 23 00:54:15.641199 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.641133    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1946d96c-c024-401f-9fc1-4f0efe2c5a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq\" (UID: \"1946d96c-c024-401f-9fc1-4f0efe2c5a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:15.742236 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.742209    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1946d96c-c024-401f-9fc1-4f0efe2c5a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq\" (UID: \"1946d96c-c024-401f-9fc1-4f0efe2c5a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:15.742857 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.742824    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1946d96c-c024-401f-9fc1-4f0efe2c5a9e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq\" (UID: \"1946d96c-c024-401f-9fc1-4f0efe2c5a9e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:15.797714 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.797690    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:15.937772 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:15.937745    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"]
Jul 23 00:54:15.938753 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:54:15.938730    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1946d96c_c024_401f_9fc1_4f0efe2c5a9e.slice/crio-3209b26b00b87782cde1ea629d062823864c3a6acb0880fb1deafeea9f908444 WatchSource:0}: Error finding container 3209b26b00b87782cde1ea629d062823864c3a6acb0880fb1deafeea9f908444: Status 404 returned error can't find the container with id 3209b26b00b87782cde1ea629d062823864c3a6acb0880fb1deafeea9f908444
Jul 23 00:54:16.075638 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:16.075612    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq" event={"ID":"1946d96c-c024-401f-9fc1-4f0efe2c5a9e","Type":"ContainerStarted","Data":"3209b26b00b87782cde1ea629d062823864c3a6acb0880fb1deafeea9f908444"}
Jul 23 00:54:16.076856 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:16.076823    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:16.076942 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:16.076822    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adf999308b2f2b12da4abcf44da6679bafad866e9a598040e769e8c29d0343da"
Jul 23 00:54:16.086541 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:16.086511    2558 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7694-139842433054736\" is forbidden: User \"system:node:ip-10-0-135-163.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-135-163.ec2.internal' and this object" podUID="a29a8e36-8ad9-48c0-adea-650195f39943" pod="opendatahub/test-curl-7694-139842433054736"
Jul 23 00:54:16.740995 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:16.740958    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a29a8e36-8ad9-48c0-adea-650195f39943" path="/var/lib/kubelet/pods/a29a8e36-8ad9-48c0-adea-650195f39943/volumes"
Jul 23 00:54:17.085022 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:17.084991    2558 generic.go:356] "Generic (PLEG): container finished" podID="1946d96c-c024-401f-9fc1-4f0efe2c5a9e" containerID="cc4fb6ba519d19115944b63a51b6f3125522f08291ff1186b53862df0ff1427a" exitCode=0
Jul 23 00:54:17.085165 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:17.085074    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq" event={"ID":"1946d96c-c024-401f-9fc1-4f0efe2c5a9e","Type":"ContainerDied","Data":"cc4fb6ba519d19115944b63a51b6f3125522f08291ff1186b53862df0ff1427a"}
Jul 23 00:54:18.223090 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:18.223062    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:18.366418 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:18.366348    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1946d96c-c024-401f-9fc1-4f0efe2c5a9e-maas-api-service-ca\") pod \"1946d96c-c024-401f-9fc1-4f0efe2c5a9e\" (UID: \"1946d96c-c024-401f-9fc1-4f0efe2c5a9e\") "
Jul 23 00:54:18.366668 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:18.366648    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1946d96c-c024-401f-9fc1-4f0efe2c5a9e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1946d96c-c024-401f-9fc1-4f0efe2c5a9e" (UID: "1946d96c-c024-401f-9fc1-4f0efe2c5a9e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:54:18.467518 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:18.467491    2558 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1946d96c-c024-401f-9fc1-4f0efe2c5a9e-maas-api-service-ca\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:54:19.099156 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:19.099130    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"
Jul 23 00:54:19.099296 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:19.099129    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq" event={"ID":"1946d96c-c024-401f-9fc1-4f0efe2c5a9e","Type":"ContainerDied","Data":"3209b26b00b87782cde1ea629d062823864c3a6acb0880fb1deafeea9f908444"}
Jul 23 00:54:19.099296 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:19.099259    2558 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3209b26b00b87782cde1ea629d062823864c3a6acb0880fb1deafeea9f908444"
Jul 23 00:54:21.520392 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:21.520357    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:29.719669 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:29.719636    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:30.613283 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:30.613252    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"]
Jul 23 00:54:30.632037 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:30.632010    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-59a77d-98fvq"]
Jul 23 00:54:30.736519 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:30.736492    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1946d96c-c024-401f-9fc1-4f0efe2c5a9e" path="/var/lib/kubelet/pods/1946d96c-c024-401f-9fc1-4f0efe2c5a9e/volumes"
Jul 23 00:54:31.533218 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:31.533186    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:41.587265 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:41.587230    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:44.670721 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:44.670686    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:51.643330 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:51.643295    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:54:59.350138 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:59.350101    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"]
Jul 23 00:54:59.350601 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:54:59.350429    2558 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy" containerID="cri-o://918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2" gracePeriod=30
Jul 23 00:54:59.734719 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:54:59.734639    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:55:01.653554 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:55:01.653519    2558 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod06cf0266_a499_46da_8f27_fb517871d8bf.slice/crio-41348a230f71204328fd818b07b636fcddd30ef4dcbbd1c907f4c9958597e2be.scope\": RecentStats: unable to find data in memory cache]"
Jul 23 00:55:04.610920 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.610897    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:55:04.753217 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753139    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8pjj\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-kube-api-access-v8pjj\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753217 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753195    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-istio-token\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753244    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8affd781-851e-4997-b862-08dedd2e8479-istiod-ca-cert\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753278    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-socket\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753431 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753396    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-credential-socket\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753589 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753465    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-certs\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753589 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753499    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-data\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753589 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753522    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8affd781-851e-4997-b862-08dedd2e8479-istio-podinfo\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753589 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753536    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:04.753804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753630    2558 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-envoy\") pod \"8affd781-851e-4997-b862-08dedd2e8479\" (UID: \"8affd781-851e-4997-b862-08dedd2e8479\") "
Jul 23 00:55:04.753804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753653    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:04.753804 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753712    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8affd781-851e-4997-b862-08dedd2e8479-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 23 00:55:04.753996 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753947    2558 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8affd781-851e-4997-b862-08dedd2e8479-istiod-ca-cert\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.753996 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753966    2558 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-socket\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.753996 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.753979    2558 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-credential-socket\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.754109 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.754052    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:04.754408 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.754383    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-data" (OuterVolumeSpecName: "istio-data") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:04.755877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.755821    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8affd781-851e-4997-b862-08dedd2e8479-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 23 00:55:04.755877 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.755852    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-kube-api-access-v8pjj" (OuterVolumeSpecName: "kube-api-access-v8pjj") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "kube-api-access-v8pjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:55:04.756022 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.755885    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-istio-token" (OuterVolumeSpecName: "istio-token") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 23 00:55:04.756022 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.755964    2558 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8affd781-851e-4997-b862-08dedd2e8479" (UID: "8affd781-851e-4997-b862-08dedd2e8479"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 23 00:55:04.855345 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.855322    2558 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-istio-token\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.855345 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.855344    2558 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-workload-certs\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.855481 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.855356    2558 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-data\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.855481 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.855364    2558 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8affd781-851e-4997-b862-08dedd2e8479-istio-podinfo\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.855481 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.855372    2558 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8affd781-851e-4997-b862-08dedd2e8479-istio-envoy\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:04.855481 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:04.855380    2558 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8pjj\" (UniqueName: \"kubernetes.io/projected/8affd781-851e-4997-b862-08dedd2e8479-kube-api-access-v8pjj\") on node \"ip-10-0-135-163.ec2.internal\" DevicePath \"\""
Jul 23 00:55:05.369730 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.369695    2558 generic.go:356] "Generic (PLEG): container finished" podID="8affd781-851e-4997-b862-08dedd2e8479" containerID="918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2" exitCode=0
Jul 23 00:55:05.369938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.369752    2558 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"
Jul 23 00:55:05.369938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.369786    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" event={"ID":"8affd781-851e-4997-b862-08dedd2e8479","Type":"ContainerDied","Data":"918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2"}
Jul 23 00:55:05.369938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.369867    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq" event={"ID":"8affd781-851e-4997-b862-08dedd2e8479","Type":"ContainerDied","Data":"a43456127289e0acbaa99900355c0756d8eefb12818d7f6447f3346abd7506f5"}
Jul 23 00:55:05.369938 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.369902    2558 scope.go:117] "RemoveContainer" containerID="918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2"
Jul 23 00:55:05.380579 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.380564    2558 scope.go:117] "RemoveContainer" containerID="d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"
Jul 23 00:55:05.391030 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.391007    2558 scope.go:117] "RemoveContainer" containerID="918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2"
Jul 23 00:55:05.391257 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:55:05.391240    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2\": container with ID starting with 918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2 not found: ID does not exist" containerID="918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2"
Jul 23 00:55:05.391312 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.391264    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2"} err="failed to get container status \"918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2\": rpc error: code = NotFound desc = could not find container \"918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2\": container with ID starting with 918e04f001f4d732ba9f312bfe7e993bb73cf382b19ab631c9f4c6c5cbe09cf2 not found: ID does not exist"
Jul 23 00:55:05.391312 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.391281    2558 scope.go:117] "RemoveContainer" containerID="d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"
Jul 23 00:55:05.391528 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:55:05.391512    2558 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c\": container with ID starting with d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c not found: ID does not exist" containerID="d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"
Jul 23 00:55:05.391568 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.391533    2558 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c"} err="failed to get container status \"d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c\": rpc error: code = NotFound desc = could not find container \"d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c\": container with ID starting with d6e76db508d641852af77b62cb70b0271f4390b4efee129ef683dd86382ea41c not found: ID does not exist"
Jul 23 00:55:05.394880 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.394856    2558 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"]
Jul 23 00:55:05.398810 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:05.398787    2558 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-d50941-openshift-default-5978cf4675-k6mkq"]
Jul 23 00:55:06.736421 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:06.736391    2558 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8affd781-851e-4997-b862-08dedd2e8479" path="/var/lib/kubelet/pods/8affd781-851e-4997-b862-08dedd2e8479/volumes"
Jul 23 00:55:06.796553 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:06.796531    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86b98fc76f-hdt4v_2ed5c242-d656-4ccc-a10a-26a9d7538ae5/copy-manifests/0.log"
Jul 23 00:55:06.806380 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:06.806361    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86b98fc76f-hdt4v_2ed5c242-d656-4ccc-a10a-26a9d7538ae5/manager/0.log"
Jul 23 00:55:07.300778 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:07.300745    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29746110-8jcxr_5d234332-9be7-49a5-a2be-3e27459ee4ac/cleanup/0.log"
Jul 23 00:55:08.381941 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.381905    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh_f12feec1-0eb2-4580-9ce4-1f8a0ab96826/util/0.log"
Jul 23 00:55:08.389455 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.389427    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh_f12feec1-0eb2-4580-9ce4-1f8a0ab96826/pull/0.log"
Jul 23 00:55:08.396178 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.396152    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh_f12feec1-0eb2-4580-9ce4-1f8a0ab96826/extract/0.log"
Jul 23 00:55:08.512646 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.512622    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4_ca48f794-daa3-41cf-9f07-56fb81211503/util/0.log"
Jul 23 00:55:08.519603 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.519583    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4_ca48f794-daa3-41cf-9f07-56fb81211503/pull/0.log"
Jul 23 00:55:08.526410 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.526372    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4_ca48f794-daa3-41cf-9f07-56fb81211503/extract/0.log"
Jul 23 00:55:08.643706 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.643645    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8_d729d699-83ae-42ee-8c4a-7fafe7e59d90/util/0.log"
Jul 23 00:55:08.650805 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.650789    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8_d729d699-83ae-42ee-8c4a-7fafe7e59d90/pull/0.log"
Jul 23 00:55:08.656578 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.656545    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8_d729d699-83ae-42ee-8c4a-7fafe7e59d90/extract/0.log"
Jul 23 00:55:08.778327 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.778304    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw_d1a95112-5955-4326-9e24-2ef5e1164095/util/0.log"
Jul 23 00:55:08.785375 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.785354    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw_d1a95112-5955-4326-9e24-2ef5e1164095/pull/0.log"
Jul 23 00:55:08.791768 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:08.791752    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw_d1a95112-5955-4326-9e24-2ef5e1164095/extract/0.log"
Jul 23 00:55:09.158253 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:09.158221    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mr7kr_af8d373b-36f5-419a-878a-77d992b312c3/manager/0.log"
Jul 23 00:55:09.273216 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:09.273197    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4gvfd_2dfc6ebd-a7ef-4969-9a20-42c08a5f0380/kuadrant-console-plugin/0.log"
Jul 23 00:55:09.404953 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:09.404930    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lzvg7_eda5c88d-76ed-4cbe-bf15-2e80aa228982/registry-server/0.log"
Jul 23 00:55:09.546748 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:09.546680    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-96nvp_d3e20902-9c43-4ca3-8c48-79cf3d2d8e71/manager/0.log"
Jul 23 00:55:09.807681 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:09.807660    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2rvct_49377f17-0c9d-43ae-85fb-2d85904f8ef6/manager/0.log"
Jul 23 00:55:10.194605 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:10.194525    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wgsm4_cf4c1ae8-bd08-46ff-8c1e-71a264c49963/istio-proxy/0.log"
Jul 23 00:55:10.619531 ip-10-0-135-163 kubenswrapper[2558]: E0723 00:55:10.619500    2558 fsHandler.go:119] failed to collect filesystem stats - rootDiskErr: could not stat "/var/lib/containers/storage/overlay/16313fd93eb341d6d78379fb9f227c1b2fc0e126a876c7f0b27b5d312920b277/diff" to get inode usage: stat /var/lib/containers/storage/overlay/16313fd93eb341d6d78379fb9f227c1b2fc0e126a876c7f0b27b5d312920b277/diff: no such file or directory, extraDiskErr: could not stat "/var/log/pods/ai-tenant-e2e-shared-b-d50941_rate-test-model-d50941-kserve-747fdfc7f-48zh9_06cf0266-a499-46da-8f27-fb517871d8bf/main/0.log" to get inode usage: stat /var/log/pods/ai-tenant-e2e-shared-b-d50941_rate-test-model-d50941-kserve-747fdfc7f-48zh9_06cf0266-a499-46da-8f27-fb517871d8bf/main/0.log: no such file or directory
Jul 23 00:55:11.083921 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:11.083885    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56d94dd6b-z49t2_bddc0a11-3d3b-4b42-90a0-a2715c845942/router/0.log"
Jul 23 00:55:11.564331 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:11.564307    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-v8m29_30fe6e84-f3b6-4556-b9eb-19b15f1e5e43/main/0.log"
Jul 23 00:55:11.935911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:11.935810    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-bj8jq_d43cfa60-497d-4048-949a-998bd6ee22e0/main/0.log"
Jul 23 00:55:17.493159 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493121    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5lwts/must-gather-ktv7r"]
Jul 23 00:55:17.493529 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493510    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy"
Jul 23 00:55:17.493529 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493521    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy"
Jul 23 00:55:17.493602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493542    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1946d96c-c024-401f-9fc1-4f0efe2c5a9e" containerName="revoke-keys"
Jul 23 00:55:17.493602 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493548    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="1946d96c-c024-401f-9fc1-4f0efe2c5a9e" containerName="revoke-keys"
Jul 23 00:55:17.493667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493604    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy"
Jul 23 00:55:17.493667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493613    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy"
Jul 23 00:55:17.493667 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493621    2558 memory_manager.go:368] "RemoveStaleState removing state" podUID="1946d96c-c024-401f-9fc1-4f0efe2c5a9e" containerName="revoke-keys"
Jul 23 00:55:17.493760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493689    2558 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy"
Jul 23 00:55:17.493760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.493695    2558 state_mem.go:107] "Deleted CPUSet assignment" podUID="8affd781-851e-4997-b862-08dedd2e8479" containerName="istio-proxy"
Jul 23 00:55:17.496801 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.496781    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.499206 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.499183    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5lwts\"/\"openshift-service-ca.crt\""
Jul 23 00:55:17.500072 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.500023    2558 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5lwts\"/\"default-dockercfg-mrwpc\""
Jul 23 00:55:17.500189 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.500049    2558 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5lwts\"/\"kube-root-ca.crt\""
Jul 23 00:55:17.508064 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.508040    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lwts/must-gather-ktv7r"]
Jul 23 00:55:17.657469 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.657440    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5l9\" (UniqueName: \"kubernetes.io/projected/6f6e101f-f98f-4a12-b464-d36220c81874-kube-api-access-sl5l9\") pod \"must-gather-ktv7r\" (UID: \"6f6e101f-f98f-4a12-b464-d36220c81874\") " pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.657581 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.657509    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6f6e101f-f98f-4a12-b464-d36220c81874-must-gather-output\") pod \"must-gather-ktv7r\" (UID: \"6f6e101f-f98f-4a12-b464-d36220c81874\") " pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.758165 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.758107    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6f6e101f-f98f-4a12-b464-d36220c81874-must-gather-output\") pod \"must-gather-ktv7r\" (UID: \"6f6e101f-f98f-4a12-b464-d36220c81874\") " pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.758165 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.758158    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5l9\" (UniqueName: \"kubernetes.io/projected/6f6e101f-f98f-4a12-b464-d36220c81874-kube-api-access-sl5l9\") pod \"must-gather-ktv7r\" (UID: \"6f6e101f-f98f-4a12-b464-d36220c81874\") " pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.758460 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.758441    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/6f6e101f-f98f-4a12-b464-d36220c81874-must-gather-output\") pod \"must-gather-ktv7r\" (UID: \"6f6e101f-f98f-4a12-b464-d36220c81874\") " pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.767898 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.767874    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5l9\" (UniqueName: \"kubernetes.io/projected/6f6e101f-f98f-4a12-b464-d36220c81874-kube-api-access-sl5l9\") pod \"must-gather-ktv7r\" (UID: \"6f6e101f-f98f-4a12-b464-d36220c81874\") " pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.806051 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.806024    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lwts/must-gather-ktv7r"
Jul 23 00:55:17.939869 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:17.939821    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lwts/must-gather-ktv7r"]
Jul 23 00:55:17.941875 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:55:17.941849    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6f6e101f_f98f_4a12_b464_d36220c81874.slice/crio-365a58498c78cdb9ef31525ddcfce829e9d971445467ac73c2c76f49118d52df WatchSource:0}: Error finding container 365a58498c78cdb9ef31525ddcfce829e9d971445467ac73c2c76f49118d52df: Status 404 returned error can't find the container with id 365a58498c78cdb9ef31525ddcfce829e9d971445467ac73c2c76f49118d52df
Jul 23 00:55:18.452121 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:18.452091    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lwts/must-gather-ktv7r" event={"ID":"6f6e101f-f98f-4a12-b464-d36220c81874","Type":"ContainerStarted","Data":"365a58498c78cdb9ef31525ddcfce829e9d971445467ac73c2c76f49118d52df"}
Jul 23 00:55:19.462108 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:19.462066    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lwts/must-gather-ktv7r" event={"ID":"6f6e101f-f98f-4a12-b464-d36220c81874","Type":"ContainerStarted","Data":"6da8839ca193ba730848299217125936d49f1ec61cd7e4b72f2f3006ab9449dc"}
Jul 23 00:55:19.462610 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:19.462124    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lwts/must-gather-ktv7r" event={"ID":"6f6e101f-f98f-4a12-b464-d36220c81874","Type":"ContainerStarted","Data":"d3d940067b5a1dad60919e8f4ba7286dcbf356d6d558d23c1a6255a1ce625315"}
Jul 23 00:55:19.483218 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:19.483153    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5lwts/must-gather-ktv7r" podStartSLOduration=1.5916128939999998 podStartE2EDuration="2.483133853s" totalImagesPullingTime="891.520959ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:55:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:55:19.482184047 +0000 UTC m=+3069.206499255" watchObservedRunningTime="2026-07-23 00:55:19.483133853 +0000 UTC m=+3069.207449038"
Jul 23 00:55:20.581458 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:20.581427    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-v79jt_07bb7baf-47aa-4d60-9bf1-881afa525a35/global-pull-secret-syncer/0.log"
Jul 23 00:55:20.712573 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:20.712532    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-l8l9p_2fc610a8-6e74-4c7f-8822-9db2cf74bda0/konnectivity-agent/0.log"
Jul 23 00:55:20.746366 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:20.746331    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-163.ec2.internal_bfa0d9d9a7caac4bf13571fee88d8afb/haproxy/0.log"
Jul 23 00:55:24.989068 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:24.989034    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh_f12feec1-0eb2-4580-9ce4-1f8a0ab96826/extract/0.log"
Jul 23 00:55:25.019520 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.019490    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh_f12feec1-0eb2-4580-9ce4-1f8a0ab96826/util/0.log"
Jul 23 00:55:25.047645 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.047601    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759d9trh_f12feec1-0eb2-4580-9ce4-1f8a0ab96826/pull/0.log"
Jul 23 00:55:25.084417 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.084334    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4_ca48f794-daa3-41cf-9f07-56fb81211503/extract/0.log"
Jul 23 00:55:25.113457 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.113427    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4_ca48f794-daa3-41cf-9f07-56fb81211503/util/0.log"
Jul 23 00:55:25.145071 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.145039    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0hc4w4_ca48f794-daa3-41cf-9f07-56fb81211503/pull/0.log"
Jul 23 00:55:25.183273 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.183223    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8_d729d699-83ae-42ee-8c4a-7fafe7e59d90/extract/0.log"
Jul 23 00:55:25.223945 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.223908    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8_d729d699-83ae-42ee-8c4a-7fafe7e59d90/util/0.log"
Jul 23 00:55:25.252292 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.252259    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73d7jg8_d729d699-83ae-42ee-8c4a-7fafe7e59d90/pull/0.log"
Jul 23 00:55:25.291740 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.291708    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw_d1a95112-5955-4326-9e24-2ef5e1164095/extract/0.log"
Jul 23 00:55:25.329174 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.329127    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw_d1a95112-5955-4326-9e24-2ef5e1164095/util/0.log"
Jul 23 00:55:25.365489 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.365388    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16pgrw_d1a95112-5955-4326-9e24-2ef5e1164095/pull/0.log"
Jul 23 00:55:25.857473 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.857442    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-mr7kr_af8d373b-36f5-419a-878a-77d992b312c3/manager/0.log"
Jul 23 00:55:25.899330 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.899304    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-4gvfd_2dfc6ebd-a7ef-4969-9a20-42c08a5f0380/kuadrant-console-plugin/0.log"
Jul 23 00:55:25.955037 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:25.955004    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-lzvg7_eda5c88d-76ed-4cbe-bf15-2e80aa228982/registry-server/0.log"
Jul 23 00:55:26.216111 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:26.216009    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-96nvp_d3e20902-9c43-4ca3-8c48-79cf3d2d8e71/manager/0.log"
Jul 23 00:55:26.361913 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:26.361829    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2rvct_49377f17-0c9d-43ae-85fb-2d85904f8ef6/manager/0.log"
Jul 23 00:55:28.012315 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.012278    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/alertmanager/0.log"
Jul 23 00:55:28.045909 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.045881    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/config-reloader/0.log"
Jul 23 00:55:28.075147 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.075121    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/kube-rbac-proxy-web/0.log"
Jul 23 00:55:28.136808 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.136776    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/kube-rbac-proxy/0.log"
Jul 23 00:55:28.195792 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.195763    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/kube-rbac-proxy-metric/0.log"
Jul 23 00:55:28.229637 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.229612    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/prom-label-proxy/0.log"
Jul 23 00:55:28.256782 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.256750    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e35a6dc5-45d5-424d-8a0c-8d9712cbb10a/init-config-reloader/0.log"
Jul 23 00:55:28.313746 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.313714    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-jdss5_ca6d1813-9908-48e9-9c00-1cb34c39b3c9/cluster-monitoring-operator/0.log"
Jul 23 00:55:28.434271 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.434237    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-b76f97fd-bdsbj_dbbcbdac-a55e-481f-99d6-b40966163542/metrics-server/0.log"
Jul 23 00:55:28.504425 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.504371    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2w2sb_265e6475-c359-4a73-b86c-905041a290bf/node-exporter/0.log"
Jul 23 00:55:28.526742 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.526704    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2w2sb_265e6475-c359-4a73-b86c-905041a290bf/kube-rbac-proxy/0.log"
Jul 23 00:55:28.551636 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.551602    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2w2sb_265e6475-c359-4a73-b86c-905041a290bf/init-textfile/0.log"
Jul 23 00:55:28.740543 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.740516    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8t6k2_b12e1b4d-0f8e-478a-b821-a7d01d92cb78/kube-rbac-proxy-main/0.log"
Jul 23 00:55:28.764617 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.764587    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8t6k2_b12e1b4d-0f8e-478a-b821-a7d01d92cb78/kube-rbac-proxy-self/0.log"
Jul 23 00:55:28.787821 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:28.787794    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-8t6k2_b12e1b4d-0f8e-478a-b821-a7d01d92cb78/openshift-state-metrics/0.log"
Jul 23 00:55:29.055012 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.054904    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-pt7bw_ab615151-090f-437f-a794-839373039819/prometheus-operator-admission-webhook/0.log"
Jul 23 00:55:29.095525 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.095495    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cfdf887f6-bpv9g_16d07331-b5a0-442b-8281-0a987f01c56a/telemeter-client/0.log"
Jul 23 00:55:29.119560 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.119525    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cfdf887f6-bpv9g_16d07331-b5a0-442b-8281-0a987f01c56a/reload/0.log"
Jul 23 00:55:29.142413 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.142378    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6cfdf887f6-bpv9g_16d07331-b5a0-442b-8281-0a987f01c56a/kube-rbac-proxy/0.log"
Jul 23 00:55:29.171500 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.171472    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7885b4b854-lrjzb_69584887-f0ad-4f3f-8075-fb4c9a8e6856/thanos-query/0.log"
Jul 23 00:55:29.192415 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.192389    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7885b4b854-lrjzb_69584887-f0ad-4f3f-8075-fb4c9a8e6856/kube-rbac-proxy-web/0.log"
Jul 23 00:55:29.216063 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.216024    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7885b4b854-lrjzb_69584887-f0ad-4f3f-8075-fb4c9a8e6856/kube-rbac-proxy/0.log"
Jul 23 00:55:29.239551 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.239524    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7885b4b854-lrjzb_69584887-f0ad-4f3f-8075-fb4c9a8e6856/prom-label-proxy/0.log"
Jul 23 00:55:29.266182 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.266157    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7885b4b854-lrjzb_69584887-f0ad-4f3f-8075-fb4c9a8e6856/kube-rbac-proxy-rules/0.log"
Jul 23 00:55:29.289253 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:29.289225    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7885b4b854-lrjzb_69584887-f0ad-4f3f-8075-fb4c9a8e6856/kube-rbac-proxy-metrics/0.log"
Jul 23 00:55:30.185621 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.185579    2558 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"]
Jul 23 00:55:30.192977 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.192946    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.198385 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.198352    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"]
Jul 23 00:55:30.300051 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.300007    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-lib-modules\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.300252 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.300113    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn9xq\" (UniqueName: \"kubernetes.io/projected/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-kube-api-access-wn9xq\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.300252 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.300181    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-sys\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.300252 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.300210    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-podres\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.300441 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.300267    2558 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-proc\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401402 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401371    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-sys\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401402 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401408    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-podres\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401629 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401525    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-proc\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401629 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401526    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-sys\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401629 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401550    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-lib-modules\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401766 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401638    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-lib-modules\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401766 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401649    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-podres\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401766 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401664    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-proc\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.401766 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.401709    2558 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn9xq\" (UniqueName: \"kubernetes.io/projected/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-kube-api-access-wn9xq\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.410334 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.410309    2558 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn9xq\" (UniqueName: \"kubernetes.io/projected/673d3a9c-35d0-4e64-a20f-5f9d66b3e274-kube-api-access-wn9xq\") pod \"perf-node-gather-daemonset-pxr8p\" (UID: \"673d3a9c-35d0-4e64-a20f-5f9d66b3e274\") " pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.505887 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.505773    2558 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:30.701014 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:30.700973    2558 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"]
Jul 23 00:55:30.705373 ip-10-0-135-163 kubenswrapper[2558]: W0723 00:55:30.705337    2558 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod673d3a9c_35d0_4e64_a20f_5f9d66b3e274.slice/crio-a77649092f7dd04a4146700de82b8a9632824874b75a9e56764cd3f8bac6a638 WatchSource:0}: Error finding container a77649092f7dd04a4146700de82b8a9632824874b75a9e56764cd3f8bac6a638: Status 404 returned error can't find the container with id a77649092f7dd04a4146700de82b8a9632824874b75a9e56764cd3f8bac6a638
Jul 23 00:55:31.568308 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:31.568274    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p" event={"ID":"673d3a9c-35d0-4e64-a20f-5f9d66b3e274","Type":"ContainerStarted","Data":"414ff81e588b2e2a9ae07a760b098fea53c51067537c2be635857c684ff26b59"}
Jul 23 00:55:31.568308 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:31.568323    2558 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p" event={"ID":"673d3a9c-35d0-4e64-a20f-5f9d66b3e274","Type":"ContainerStarted","Data":"a77649092f7dd04a4146700de82b8a9632824874b75a9e56764cd3f8bac6a638"}
Jul 23 00:55:31.568760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:31.568410    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:31.586054 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:31.586010    2558 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p" podStartSLOduration=1.585995666 podStartE2EDuration="1.585995666s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:55:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:55:31.585302754 +0000 UTC m=+3081.309617965" watchObservedRunningTime="2026-07-23 00:55:31.585995666 +0000 UTC m=+3081.310310851"
Jul 23 00:55:31.599751 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:31.599724    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5787fc7c8-5xbpf_6ba9964f-ad29-47f3-b715-a5f1adf51653/console/0.log"
Jul 23 00:55:32.209343 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:32.209308    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-9nkpn_65b77b76-07fb-4d85-9a53-a9e91029aae0/volume-data-source-validator/0.log"
Jul 23 00:55:33.421619 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:33.421588    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ttkgf_23918e6b-3a8a-4f61-98c5-dc0f21cb1b23/dns/0.log"
Jul 23 00:55:33.444263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:33.444232    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ttkgf_23918e6b-3a8a-4f61-98c5-dc0f21cb1b23/kube-rbac-proxy/0.log"
Jul 23 00:55:33.521799 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:33.521770    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fcgjk_7509dbfc-f6e2-46ea-a785-32e381851a89/dns-node-resolver/0.log"
Jul 23 00:55:34.057443 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:34.057416    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29746080-c9kg2_1b718f23-82f5-4b53-9f6b-e17f42514a82/image-pruner/0.log"
Jul 23 00:55:34.122605 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:34.122574    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5sfgf_78c775d2-bca1-4090-837a-078c4a09fb98/node-ca/0.log"
Jul 23 00:55:35.131713 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:35.131681    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wgsm4_cf4c1ae8-bd08-46ff-8c1e-71a264c49963/istio-proxy/0.log"
Jul 23 00:55:35.452309 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:35.452223    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56d94dd6b-z49t2_bddc0a11-3d3b-4b42-90a0-a2715c845942/router/0.log"
Jul 23 00:55:36.157675 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:36.157649    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-l995d_5a60f4c3-d54a-42ef-93ef-899e4e819daf/serve-healthcheck-canary/0.log"
Jul 23 00:55:36.640800 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:36.640762    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-p5kxt_29935393-8973-47ec-9c30-b7a21f0e270f/insights-operator/0.log"
Jul 23 00:55:36.643663 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:36.643631    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-p5kxt_29935393-8973-47ec-9c30-b7a21f0e270f/insights-operator/1.log"
Jul 23 00:55:36.673082 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:36.673051    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-csg5h_7902e54a-d09e-446d-b377-3ddf56e3624e/kube-rbac-proxy/0.log"
Jul 23 00:55:36.698760 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:36.698734    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-csg5h_7902e54a-d09e-446d-b377-3ddf56e3624e/exporter/0.log"
Jul 23 00:55:36.722261 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:36.722232    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-csg5h_7902e54a-d09e-446d-b377-3ddf56e3624e/extractor/0.log"
Jul 23 00:55:37.589750 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:37.589726    2558 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5lwts/perf-node-gather-daemonset-pxr8p"
Jul 23 00:55:39.904323 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:39.904291    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86b98fc76f-hdt4v_2ed5c242-d656-4ccc-a10a-26a9d7538ae5/manager/0.log"
Jul 23 00:55:39.942066 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:39.942041    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-86b98fc76f-hdt4v_2ed5c242-d656-4ccc-a10a-26a9d7538ae5/copy-manifests/0.log"
Jul 23 00:55:40.653440 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:40.653406    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wgsm4_cf4c1ae8-bd08-46ff-8c1e-71a264c49963/istio-proxy/0.log"
Jul 23 00:55:40.916098 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:40.916009    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56d94dd6b-z49t2_bddc0a11-3d3b-4b42-90a0-a2715c845942/router/0.log"
Jul 23 00:55:41.225322 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:41.225233    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7dcccd7c7c-49mfj_e8ddf83f-0651-4f88-afa0-a2348cec97ac/manager/0.log"
Jul 23 00:55:41.286263 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:41.286233    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-72grd_029e47fc-9c1f-44c9-964c-95cd1093b90b/openshift-lws-operator/0.log"
Jul 23 00:55:42.131551 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:42.131467    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4wgsm4_cf4c1ae8-bd08-46ff-8c1e-71a264c49963/istio-proxy/0.log"
Jul 23 00:55:42.373616 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:42.373585    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-56d94dd6b-z49t2_bddc0a11-3d3b-4b42-90a0-a2715c845942/router/0.log"
Jul 23 00:55:47.550239 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:47.550213    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4llx_976089ec-d588-4575-94ad-b9af93c54f82/kube-multus-additional-cni-plugins/0.log"
Jul 23 00:55:47.575057 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:47.575014    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4llx_976089ec-d588-4575-94ad-b9af93c54f82/egress-router-binary-copy/0.log"
Jul 23 00:55:47.598191 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:47.598170    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4llx_976089ec-d588-4575-94ad-b9af93c54f82/cni-plugins/0.log"
Jul 23 00:55:47.621152 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:47.621129    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4llx_976089ec-d588-4575-94ad-b9af93c54f82/bond-cni-plugin/0.log"
Jul 23 00:55:47.644911 ip-10-0-135-163 kubenswrapper[2558]: I0723 00:55:47.644874    2558 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-f4llx_976089ec-d588-4575-94ad-b9af93c54f82/routeoverride-cni/0.log"