Jul 13 21:09:21.807404 ip-10-0-131-147 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 13 21:09:21.807418 ip-10-0-131-147 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 13 21:09:21.807427 ip-10-0-131-147 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 13 21:09:21.807725 ip-10-0-131-147 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 13 21:09:32.027141 ip-10-0-131-147 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 13 21:09:32.027164 ip-10-0-131-147 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 8d51a28168eb49f79a0a6bf55be7cac1 --
Jul 13 21:11:30.873926 ip-10-0-131-147 systemd[1]: Starting Kubernetes Kubelet...
Jul 13 21:11:31.344682 ip-10-0-131-147 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:11:31.344682 ip-10-0-131-147 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 13 21:11:31.344682 ip-10-0-131-147 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:11:31.344682 ip-10-0-131-147 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 13 21:11:31.344682 ip-10-0-131-147 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 13 21:11:31.345765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.345657    2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 13 21:11:31.350227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350211    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:31.350227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350228    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350231    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350235    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350240    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350244    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350253    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350257    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350259    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350262    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350265    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350267    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350270    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350272    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350275    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350277    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350280    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350283    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350285    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350288    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350290    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:31.350294 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350293    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350296    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350299    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350301    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350304    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350308    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350311    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350314    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350317    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350319    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350323    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350325    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350328    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350330    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350333    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350335    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350338    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350341    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350343    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350345    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:31.350890 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350348    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350350    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350353    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350355    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350358    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350360    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350363    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350367    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350371    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350374    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350377    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350380    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350383    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350386    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350390    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350393    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350396    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350398    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350401    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:31.351383 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350404    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350406    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350408    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350411    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350414    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350417    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350419    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350422    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350425    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350428    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350430    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350432    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350435    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350437    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350441    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350443    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350445    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350448    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350451    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350454    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:31.351846 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350456    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350459    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350461    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350463    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350466    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350468    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350471    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350475    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350478    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350480    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350483    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350486    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350488    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350491    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350493    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350496    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350498    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350503    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.350506    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351216    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:31.352311 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351223    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351226    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351228    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351231    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351234    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351239    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351243    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351246    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351249    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351252    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351254    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351257    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351260    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351263    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351266    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351268    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351271    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351274    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351276    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:31.352792 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351280    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351282    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351285    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351288    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351290    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351294    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351297    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351300    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351302    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351305    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351307    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351311    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351313    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351316    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351318    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351321    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351323    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351326    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351328    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:31.353282 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351331    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351335    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351338    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351341    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351344    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351347    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351350    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351353    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351356    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351359    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351361    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351364    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351367    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351370    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351373    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351376    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351378    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351382    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351384    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:31.353804 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351387    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351390    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351392    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351395    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351397    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351400    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351403    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351405    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351408    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351410    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351413    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351415    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351418    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351420    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351422    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351425    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351428    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351431    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351435    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351438    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:31.354256 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351441    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351444    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351446    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351449    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351451    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351454    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351457    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351459    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351461    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351464    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351466    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351469    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351471    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351474    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351476    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351478    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351480    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351483    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351486    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351489    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:31.354752 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351491    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351886    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351891    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351894    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351897    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351900    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351903    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351905    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351909    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351911    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351915    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351919    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351921    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351924    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351926    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351929    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351932    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351934    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351937    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:31.355227 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351939    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351942    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351944    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351947    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351949    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351952    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351955    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351957    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351960    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351962    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351965    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351967    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351973    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351976    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351979    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351981    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351984    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351986    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351989    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351992    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:31.355693 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351994    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.351998    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352003    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352006    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352010    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352013    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352016    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352019    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352021    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352023    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352026    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352028    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352031    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352034    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352036    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352038    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352041    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352043    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352045    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:31.356163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352048    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352050    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352053    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352055    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352058    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352060    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352063    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352066    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352068    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352071    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352073    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352076    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352079    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352081    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352084    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352086    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352089    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352092    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352094    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352097    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:31.356632 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352099    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352101    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352104    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352106    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352111    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352115    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352119    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352122    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352125    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352127    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352130    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352132    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352134    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352137    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352139    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352142    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352144    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352147    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352149    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:31.357105 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352152    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352157    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352160    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352231    2564 flags.go:64] FLAG: --address="0.0.0.0"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352238    2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352246    2564 flags.go:64] FLAG: --anonymous-auth="true"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352250    2564 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352254    2564 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352257    2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352261    2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352266    2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352269    2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352272    2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352276    2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352279    2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352282    2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352285    2564 flags.go:64] FLAG: --cgroup-root=""
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352288    2564 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352291    2564 flags.go:64] FLAG: --client-ca-file=""
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352294    2564 flags.go:64] FLAG: --cloud-provider="external"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352297    2564 flags.go:64] FLAG: --cluster-dns="[]"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352301    2564 flags.go:64] FLAG: --cluster-domain=""
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352304    2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352308    2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 13 21:11:31.357567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352311    2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352314    2564 flags.go:64] FLAG: --container-log-max-files="5"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352318    2564 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352321    2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352325    2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352328    2564 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352332    2564 flags.go:64] FLAG: --contention-profiling="false"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352335    2564 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352338    2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352343    2564 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352346    2564 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352351    2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352354    2564 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352357    2564 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352360    2564 flags.go:64] FLAG: --enable-load-reader="false"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352363    2564 flags.go:64] FLAG: --enable-server="true"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352366    2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352371    2564 flags.go:64] FLAG: --event-burst="100"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352374    2564 flags.go:64] FLAG: --event-qps="50"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352377    2564 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352380    2564 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352383    2564 flags.go:64] FLAG: --eviction-hard=""
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352386    2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352389    2564 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352392    2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 13 21:11:31.358157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352395    2564 flags.go:64] FLAG: --eviction-soft=""
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352399    2564 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352402    2564 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352404    2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352407    2564 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352410    2564 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352413    2564 flags.go:64] FLAG: --fail-swap-on="true"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352416    2564 flags.go:64] FLAG: --feature-gates=""
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352420    2564 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352423    2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352426    2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352429    2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352431    2564 flags.go:64] FLAG: --healthz-port="10248"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352434    2564 flags.go:64] FLAG: --help="false"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352437    2564 flags.go:64] FLAG: --hostname-override="ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352441    2564 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352444    2564 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352448    2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352452    2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352455    2564 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352458    2564 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352461    2564 flags.go:64] FLAG: --image-service-endpoint=""
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352464    2564 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 13 21:11:31.358771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352466    2564 flags.go:64] FLAG: --kube-api-burst="100"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352470    2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352473    2564 flags.go:64] FLAG: --kube-api-qps="50"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352476    2564 flags.go:64] FLAG: --kube-reserved=""
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352479    2564 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352482    2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352485    2564 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352488    2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352491    2564 flags.go:64] FLAG: --lock-file=""
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352493    2564 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352496    2564 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352499    2564 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352505    2564 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352508    2564 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352511    2564 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352514    2564 flags.go:64] FLAG: --logging-format="text"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352516    2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352520    2564 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352523    2564 flags.go:64] FLAG: --manifest-url=""
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352525    2564 flags.go:64] FLAG: --manifest-url-header=""
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352530    2564 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352533    2564 flags.go:64] FLAG: --max-open-files="1000000"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352537    2564 flags.go:64] FLAG: --max-pods="110"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352542    2564 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352545    2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 13 21:11:31.359340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352548    2564 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352551    2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352555    2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352558    2564 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352561    2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352567    2564 flags.go:64] FLAG: --node-status-max-images="50"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352570    2564 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352573    2564 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352576    2564 flags.go:64] FLAG: --pod-cidr=""
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352579    2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352585    2564 flags.go:64] FLAG: --pod-manifest-path=""
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352588    2564 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352591    2564 flags.go:64] FLAG: --pods-per-core="0"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352594    2564 flags.go:64] FLAG: --port="10250"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352597    2564 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352600    2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00e6fc72e13da0741"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352603    2564 flags.go:64] FLAG: --qos-reserved=""
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352606    2564 flags.go:64] FLAG: --read-only-port="10255"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352609    2564 flags.go:64] FLAG: --register-node="true"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352625    2564 flags.go:64] FLAG: --register-with-taints=""
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352629    2564 flags.go:64] FLAG: --registry-burst="10"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352631    2564 flags.go:64] FLAG: --registry-qps="5"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352634    2564 flags.go:64] FLAG: --reserved-cpus=""
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352637    2564 flags.go:64] FLAG: --reserved-memory=""
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352641    2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 13 21:11:31.359987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352644    2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352647    2564 flags.go:64] FLAG: --rotate-certificates="false"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352650    2564 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352653    2564 flags.go:64] FLAG: --runonce="false"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352656    2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352659    2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352663    2564 flags.go:64] FLAG: --seccomp-default="false"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352666    2564 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352669    2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352672    2564 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352676    2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352679    2564 flags.go:64] FLAG: --storage-driver-password="root"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352682    2564 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352685    2564 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352687    2564 flags.go:64] FLAG: --storage-driver-user="root"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352690    2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352693    2564 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352696    2564 flags.go:64] FLAG: --system-cgroups=""
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352699    2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352704    2564 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352706    2564 flags.go:64] FLAG: --tls-cert-file=""
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352709    2564 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352713    2564 flags.go:64] FLAG: --tls-min-version=""
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352716    2564 flags.go:64] FLAG: --tls-private-key-file=""
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352719    2564 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 13 21:11:31.360972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352722    2564 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352724    2564 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352727    2564 flags.go:64] FLAG: --v="2"
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352731    2564 flags.go:64] FLAG: --version="false"
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352735    2564 flags.go:64] FLAG: --vmodule=""
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352739    2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.352742    2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352854    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352858    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352861    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352864    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352867    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352870    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352874    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352879    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352882    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352885    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352888    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352891    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352893    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352896    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352898    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:31.362044 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352901    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352904    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352906    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352909    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352912    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352915    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352917    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352920    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352922    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352924    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352927    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352929    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352932    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352935    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352937    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352940    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352942    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352945    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352947    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352950    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:31.362993 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352952    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352955    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352957    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352961    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352965    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352969    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352972    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352975    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352978    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352981    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352984    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352987    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352989    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352992    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352995    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.352997    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353000    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353002    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353005    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:31.363855 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353008    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353010    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353013    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353015    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353018    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353020    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353023    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353025    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353028    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353030    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353033    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353035    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353038    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353040    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353043    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353045    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353049    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353054    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353057    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353059    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:31.364437 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353062    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353065    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353069    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353072    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353076    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353079    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353082    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353085    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353088    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353090    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353093    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353095    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353098    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353100    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353103    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353105    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353108    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353110    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353113    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:31.365176 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353115    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353118    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353122    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353125    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353128    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.353131    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.354345    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.362521    2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.362541    2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.362568    2564 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.362583    2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362688    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362696    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362701    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:31.365662 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362706    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362713    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362720    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362724    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362729    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362734    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362738    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362742    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362747    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362754    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362760    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362765    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362769    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362774    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362778    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362782    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362786    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362791    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362796    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:31.366062 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362800    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362805    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362809    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362813    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362817    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362821    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362827    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362834    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362839    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362843    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362847    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362851    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362855    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362860    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362864    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362868    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362872    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362876    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362881    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362885    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:31.366591 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362889    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362893    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362897    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362901    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362906    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362910    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362915    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362919    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362923    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362927    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362931    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362935    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362939    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362943    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362947    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362951    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362955    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362960    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362964    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362969    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:31.367429 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362973    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362977    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362982    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362986    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362989    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362994    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.362997    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363001    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363005    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363009    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363013    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363017    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363021    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363026    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363030    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363035    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363039    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363044    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363049    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363053    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:31.368049 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363058    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363061    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363065    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363070    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363074    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363078    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363083    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363087    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363091    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363095    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363099    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363104    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363109    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363113    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363118    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363122    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:31.368584 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363127    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.363135    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363313    2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363322    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363327    2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363332    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363336    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363340    2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363345    2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363350    2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363354    2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363359    2564 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363364    2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363368    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363374    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 13 21:11:31.369003 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363378    2564 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363383    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363387    2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363392    2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363396    2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363402    2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363408    2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363412    2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363416    2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363420    2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363425    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363429    2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363433    2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363438    2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363442    2564 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363446    2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363450    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363454    2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363458    2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 13 21:11:31.369361 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363463    2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363467    2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363471    2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363475    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363479    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363483    2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363488    2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363492    2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363496    2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363500    2564 feature_gate.go:328] unrecognized feature gate: Example
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363504    2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363508    2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363512    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363517    2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363522    2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363526    2564 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363531    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363535    2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363539    2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 13 21:11:31.369981 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363543    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363549    2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363555    2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363560    2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363564    2564 feature_gate.go:328] unrecognized feature gate: Example2
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363569    2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363574    2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363579    2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363584    2564 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363588    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363591    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363596    2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363600    2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363604    2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363609    2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363629    2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363634    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363638    2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363642    2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363646    2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 13 21:11:31.370485 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363650    2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363655    2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363661    2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363666    2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363670    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363674    2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363678    2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363684    2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363689    2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363693    2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363697    2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363701    2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363705    2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363710    2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363713    2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363717    2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363721    2564 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363726    2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363730    2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363734    2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 13 21:11:31.370995 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363738    2564 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363742    2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363746    2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363750    2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363754    2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363758    2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363762    2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:31.363766    2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.363774    2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.363925    2564 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.367930    2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.370340    2564 server.go:1019] "Starting client certificate rotation"
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.370439    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:11:31.371531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.370477    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 13 21:11:31.373553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.373535    2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:11:31.377064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.377041    2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 13 21:11:31.384348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.384334    2564 log.go:25] "Validated CRI v1 runtime API"
Jul 13 21:11:31.386226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.386210    2564 log.go:25] "Validated CRI v1 image API"
Jul 13 21:11:31.387799 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.387785    2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 13 21:11:31.390009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.389989    2564 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a63650a4-f238-4471-814e-63bcae7cf297:/dev/nvme0n1p3 d6de6e68-8f4b-4e50-9c06-f9a07890b04d:/dev/nvme0n1p4]
Jul 13 21:11:31.390081 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.390009    2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 13 21:11:31.396079 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.395850    2564 manager.go:216] Machine: {Timestamp:2026-07-13 21:11:31.394965528 +0000 UTC m=+0.381153587 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099507 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec26801b65724fc0264290f5a3acefb4 SystemUUID:ec26801b-6572-4fc0-2642-90f5a3acefb4 BootID:8d51a281-68eb-49f7-9a0a-6bf55be7cac1 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:27:50:b2:5b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:27:50:b2:5b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:f2:4c:8a:46:2e:56 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 13 21:11:31.396222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.396207    2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 13 21:11:31.396353 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.396324    2564 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 13 21:11:31.398222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.398205    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:11:31.398843 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.398817    2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 13 21:11:31.398982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.398846    2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-147.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 13 21:11:31.399652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.399642    2564 topology_manager.go:138] "Creating topology manager with none policy"
Jul 13 21:11:31.399684 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.399655    2564 container_manager_linux.go:309] "Creating device plugin manager"
Jul 13 21:11:31.399684 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.399662    2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:11:31.400387 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.400376    2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 13 21:11:31.401542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.401532    2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 13 21:11:31.404235 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404225    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:11:31.404352 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404343    2564 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 13 21:11:31.404386 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404372    2564 kubelet.go:486] "Attempting to sync node with API server"
Jul 13 21:11:31.404386 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404381    2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 13 21:11:31.404436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404395    2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 13 21:11:31.404436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404403    2564 kubelet.go:398] "Adding apiserver pod source"
Jul 13 21:11:31.404436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404414    2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 13 21:11:31.404781 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404769    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:11:31.404831 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.404791    2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 13 21:11:31.417924 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.417906    2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 13 21:11:31.419587 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419574    2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 13 21:11:31.419644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419605    2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 13 21:11:31.419786 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419774    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 13 21:11:31.419827 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419794    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 13 21:11:31.419827 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419800    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 13 21:11:31.419827 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419807    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 13 21:11:31.419827 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419815    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 13 21:11:31.419827 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.419821    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 13 21:11:31.420288 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.420279    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 13 21:11:31.420316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.420288    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 13 21:11:31.420316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.420295    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 13 21:11:31.420316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.420300    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 13 21:11:31.420316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.420309    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 13 21:11:31.420414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.420320    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 13 21:11:31.421297 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.421288    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 13 21:11:31.421339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.421300    2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 13 21:11:31.422128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.422109    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ggkw8"
Jul 13 21:11:31.424452 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.424439    2564 server.go:1268] "Started kubelet"
Jul 13 21:11:31.424860 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.424688    2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 13 21:11:31.424940 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.424932    2564 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 13 21:11:31.425065 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.424756    2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 13 21:11:31.425607 ip-10-0-131-147 systemd[1]: Started Kubernetes Kubelet.
Jul 13 21:11:31.426913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.426893    2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 13 21:11:31.427364 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.427334    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 13 21:11:31.427427 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.427376    2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-147.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 13 21:11:31.427478 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.427461    2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-147.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 13 21:11:31.427582 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.427565    2564 server.go:310] "Adding debug handlers to kubelet server"
Jul 13 21:11:31.429736 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.429720    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ggkw8"
Jul 13 21:11:31.431551 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.431532    2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 13 21:11:31.431956 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.431938    2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 13 21:11:31.432164 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.430995    2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-147.ec2.internal.18c1f5b675c962e0  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-147.ec2.internal,UID:ip-10-0-131-147.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-147.ec2.internal,},FirstTimestamp:2026-07-13 21:11:31.424412384 +0000 UTC m=+0.410600442,LastTimestamp:2026-07-13 21:11:31.424412384 +0000 UTC m=+0.410600442,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-147.ec2.internal,}"
Jul 13 21:11:31.432273 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.432184    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:31.432343 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.432302    2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 13 21:11:31.433390 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433373    2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 13 21:11:31.433390 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433376    2564 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 13 21:11:31.433505 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433398    2564 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 13 21:11:31.433535 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433514    2564 reconstruct.go:97] "Volume reconstruction finished"
Jul 13 21:11:31.433535 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433525    2564 reconciler.go:29] "Reconciler: start to sync state"
Jul 13 21:11:31.433690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433560    2564 factory.go:55] Registering systemd factory
Jul 13 21:11:31.433690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433575    2564 factory.go:223] Registration of the systemd container factory successfully
Jul 13 21:11:31.433818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433804    2564 factory.go:153] Registering CRI-O factory
Jul 13 21:11:31.433871 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433821    2564 factory.go:223] Registration of the crio container factory successfully
Jul 13 21:11:31.433871 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433867    2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 13 21:11:31.433972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433889    2564 factory.go:103] Registering Raw factory
Jul 13 21:11:31.433972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.433904    2564 manager.go:1196] Started watching for new ooms in manager
Jul 13 21:11:31.434257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.434245    2564 manager.go:349] Starting recovery of all containers
Jul 13 21:11:31.438537 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.438519    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:31.441070 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.441050    2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-131-147.ec2.internal\" not found" node="ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.446298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.446183    2564 manager.go:354] Recovery completed
Jul 13 21:11:31.451208 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.451196    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.454001 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.453983    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.454088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.454012    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.454088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.454022    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.454545 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.454528    2564 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 13 21:11:31.454545 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.454543    2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 13 21:11:31.454661 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.454549    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.454661 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.454561    2564 state_mem.go:36] "Initialized new in-memory state store"
Jul 13 21:11:31.455261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.455248    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.455309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.455277    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.455309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.455289    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.455679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.455668    2564 policy_none.go:49] "None policy: Start"
Jul 13 21:11:31.455725 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.455683    2564 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 13 21:11:31.455725 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.455694    2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 13 21:11:31.456744 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.456671    2564 policy_none.go:47] "Start"
Jul 13 21:11:31.493210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493193    2564 manager.go:337] "Starting Device Plugin manager"
Jul 13 21:11:31.493305 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.493230    2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 13 21:11:31.493305 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493239    2564 server.go:87] "Starting device plugin registration server"
Jul 13 21:11:31.493519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493505    2564 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 13 21:11:31.493572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493524    2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 13 21:11:31.493644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493591    2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 13 21:11:31.493713 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493700    2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 13 21:11:31.493758 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.493714    2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 13 21:11:31.495916 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.495886    2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 13 21:11:31.495987 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.495937    2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:31.593784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.593750    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.595037 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.594991    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.595037 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.595023    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.595037 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.595017    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 13 21:11:31.595037 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.595034    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.595251 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.595063    2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.596371 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.596356    2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 13 21:11:31.596426 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.596393    2564 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 13 21:11:31.596426 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.596412    2564 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 13 21:11:31.596519 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.596444    2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 13 21:11:31.598380 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.598358    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:31.602657 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.602641    2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.602740 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.602662    2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-147.ec2.internal\": node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:31.645162 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.645135    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:31.697270 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.697241    2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"]
Jul 13 21:11:31.697343 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.697318    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.698856 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.698840    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.698922 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.698873    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.698922 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.698888    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.700116 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700103    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.700202 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.700266 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700221    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.700756 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700743    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.700815 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700763    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.700815 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700772    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.700815 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700799    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.700914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700820    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.700914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.700831    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.701874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.701856    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.701969 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.701881    2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 13 21:11:31.702587 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.702573    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientMemory"
Jul 13 21:11:31.702668 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.702597    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasNoDiskPressure"
Jul 13 21:11:31.702668 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.702608    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeHasSufficientPID"
Jul 13 21:11:31.716168 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.716149    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-147.ec2.internal\" not found" node="ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.719891 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.719878    2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-147.ec2.internal\" not found" node="ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.745374 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.745354    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:31.834457 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.834430    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a2be0832c80a981e87f324caa4fb613b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal\" (UID: \"a2be0832c80a981e87f324caa4fb613b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.834457 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.834460    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2be0832c80a981e87f324caa4fb613b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal\" (UID: \"a2be0832c80a981e87f324caa4fb613b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.834649 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.834478    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ddc1535de6c0eb54f69fb18e4b2deac-config\") pod \"kube-apiserver-proxy-ip-10-0-131-147.ec2.internal\" (UID: \"0ddc1535de6c0eb54f69fb18e4b2deac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.845476 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.845426    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:31.935202 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.935166    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a2be0832c80a981e87f324caa4fb613b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal\" (UID: \"a2be0832c80a981e87f324caa4fb613b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.935202 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.935200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2be0832c80a981e87f324caa4fb613b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal\" (UID: \"a2be0832c80a981e87f324caa4fb613b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.935395 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.935217    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ddc1535de6c0eb54f69fb18e4b2deac-config\") pod \"kube-apiserver-proxy-ip-10-0-131-147.ec2.internal\" (UID: \"0ddc1535de6c0eb54f69fb18e4b2deac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.935395 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.935266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a2be0832c80a981e87f324caa4fb613b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal\" (UID: \"a2be0832c80a981e87f324caa4fb613b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.935395 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.935389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a2be0832c80a981e87f324caa4fb613b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal\" (UID: \"a2be0832c80a981e87f324caa4fb613b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.935825 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:31.935327    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0ddc1535de6c0eb54f69fb18e4b2deac-config\") pod \"kube-apiserver-proxy-ip-10-0-131-147.ec2.internal\" (UID: \"0ddc1535de6c0eb54f69fb18e4b2deac\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:31.945569 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:31.945542    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:32.018474 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.018441    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:32.023533 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.023514    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:32.046745 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.046708    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:32.147319 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.147245    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:32.247789 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.247768    2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-147.ec2.internal\" not found"
Jul 13 21:11:32.252752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.252738    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:32.332918 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.332893    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:32.342997 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.342975    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:11:32.344924 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.344910    2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal"
Jul 13 21:11:32.353775 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.353757    2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 13 21:11:32.368887 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.368869    2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 13 21:11:32.369027 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.368988    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:11:32.369106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.369016    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:11:32.369106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.369036    2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 13 21:11:32.405284 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.405207    2564 apiserver.go:52] "Watching apiserver"
Jul 13 21:11:32.422089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.422057    2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 13 21:11:32.422441 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.422419    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal","openshift-multus/multus-additional-cni-plugins-xsk6j","openshift-network-operator/iptables-alerter-9p9jl","openshift-ovn-kubernetes/ovnkube-node-6dxmp","kube-system/global-pull-secret-syncer-czhjv","kube-system/konnectivity-agent-mr9tt","kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal","openshift-dns/node-resolver-8z6tc","openshift-image-registry/node-ca-qwlrm","openshift-multus/multus-cwgjq","openshift-multus/network-metrics-daemon-q7js4","openshift-network-diagnostics/network-check-target-qs78m","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl","openshift-cluster-node-tuning-operator/tuned-cgp69"]
Jul 13 21:11:32.423984 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.423969    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.424985 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.424968    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.426760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.426742    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 13 21:11:32.426856 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.426766    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.426981 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.426960    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.427054 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.426971    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6fkct\""
Jul 13 21:11:32.427182 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.427164    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.427233 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.427182    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 13 21:11:32.427233 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.427174    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 13 21:11:32.427820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.427804    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.427912 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.427827    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 13 21:11:32.428070 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.428052    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.428199 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.428178    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vhcm9\""
Jul 13 21:11:32.428295 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.428281    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.428383 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.428357    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:32.428769 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.428753    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.429120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429105    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.429323 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429308    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.429804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429779    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 13 21:11:32.429804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429798    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-pkztb\""
Jul 13 21:11:32.429934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429817    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 13 21:11:32.429934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429838    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 13 21:11:32.429934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.429862    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 13 21:11:32.430387 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.430370    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.431358 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.431344    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.431667 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.431653    2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 13 21:11:32.431745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.431657    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 13 21:11:32.431814 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.431799    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 13 21:11:32.431853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.431836    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zzfvv\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.432490    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.432855    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.432880    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-d8kvp\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.432997    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.433771    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.434072    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.434162    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-j498h\""
Jul 13 21:11:32.434388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.434282    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 13 21:11:32.435501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.434930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:32.435501 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.435007    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:32.435501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.435275    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 13 21:11:32.435501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.435332    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 21:06:32 +0000 UTC" deadline="2028-03-28 03:50:14.908801132 +0000 UTC"
Jul 13 21:11:32.435501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.435416    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14958h38m42.473405715s"
Jul 13 21:11:32.435716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.435543    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-knsbn\""
Jul 13 21:11:32.437304 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437288    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:32.437388 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.437373    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:32.437447 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-system-cni-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.437491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437449    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-k8s-cni-cncf-io\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.437491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-systemd\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.437491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437482    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.437644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-socket-dir-parent\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.437644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437521    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-twmr6\" (UniqueName: \"kubernetes.io/projected/1bf9e6bf-545a-4ce9-b5ac-69d111087820-kube-api-access-twmr6\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.437644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:32.437644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-cnibin\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.437644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437567    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-hostroot\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.437644 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437606    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437646    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27k5k\" (UniqueName: \"kubernetes.io/projected/91276703-6b2d-40d1-8a70-1cf7d9b78570-kube-api-access-27k5k\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-cni-netd\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgvtq\" (UniqueName: \"kubernetes.io/projected/f76ffc5f-2c98-42fb-996f-a62e56716196-kube-api-access-mgvtq\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437693    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-system-cni-dir\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437761    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437794    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-ovn\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437812    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f911bedb-e01d-4f73-a947-89e2fd905115-dbus\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437827    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9kfj\" (UniqueName: \"kubernetes.io/projected/867afcc3-bfaa-457e-a67c-998900d3f853-kube-api-access-p9kfj\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437841    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-etc-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-env-overrides\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437881    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/023334c1-c6b5-4cca-b99c-d7b1572a167d-iptables-alerter-script\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.437898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437901    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-systemd-units\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437915    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-cni-bin\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/786bcfda-4260-449e-b161-d57505ba1194-agent-certs\") pod \"konnectivity-agent-mr9tt\" (UID: \"786bcfda-4260-449e-b161-d57505ba1194\") " pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/786bcfda-4260-449e-b161-d57505ba1194-konnectivity-ca\") pod \"konnectivity-agent-mr9tt\" (UID: \"786bcfda-4260-449e-b161-d57505ba1194\") " pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-slash\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437978    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-log-socket\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.437994    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438020    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-os-release\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-os-release\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438069    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovnkube-config\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438088    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-cnibin\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-multus-certs\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-etc-kubernetes\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438133    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-cni-binary-copy\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438149    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-var-lib-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438170    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovn-node-metrics-cert\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.438496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438184    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f911bedb-e01d-4f73-a947-89e2fd905115-kubelet-config\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438204    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-run-ovn-kubernetes\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438221    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz7jq\" (UniqueName: \"kubernetes.io/projected/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-kube-api-access-tz7jq\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438241    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bf9e6bf-545a-4ce9-b5ac-69d111087820-cni-binary-copy\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438263    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c66b2f39-3c48-45bc-9402-f5fbe55aecef-host\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438277    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c66b2f39-3c48-45bc-9402-f5fbe55aecef-serviceca\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-node-log\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovnkube-script-lib\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438332    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/867afcc3-bfaa-457e-a67c-998900d3f853-hosts-file\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438345    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-kubelet\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438359    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/023334c1-c6b5-4cca-b99c-d7b1572a167d-host-slash\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438375    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-conf-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438388    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-daemon-config\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438402    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kp7s2\" (UniqueName: \"kubernetes.io/projected/c66b2f39-3c48-45bc-9402-f5fbe55aecef-kube-api-access-kp7s2\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438421    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fvjk\" (UniqueName: \"kubernetes.io/projected/023334c1-c6b5-4cca-b99c-d7b1572a167d-kube-api-access-8fvjk\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438438    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-run-netns\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438453    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-cni-bin\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439094 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438467    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-cni-multus\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438490    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-kubelet\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.439866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.439866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867afcc3-bfaa-457e-a67c-998900d3f853-tmp-dir\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.439866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438555    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-cni-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438575    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-netns\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.439866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.438607    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.440129 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.440116    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.440880 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.440860    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.441103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.441088    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.441204 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.441138    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4m5k4\""
Jul 13 21:11:32.442309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.442270    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 13 21:11:32.442392 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.442323    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-kddds\""
Jul 13 21:11:32.442392 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.442330    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 13 21:11:32.442491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.442439    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 13 21:11:32.443721 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.443700    2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 13 21:11:32.460298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.460281    2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-27795"
Jul 13 21:11:32.467642 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.467609    2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-27795"
Jul 13 21:11:32.513509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.513484    2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:32.534259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.534235    2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 13 21:11:32.539866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.539837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-socket-dir-parent\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.539977 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.539879    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-twmr6\" (UniqueName: \"kubernetes.io/projected/1bf9e6bf-545a-4ce9-b5ac-69d111087820-kube-api-access-twmr6\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.539977 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.539911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:32.539977 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.539938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-cnibin\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.539977 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.539959    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-socket-dir-parent\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.540210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.539966    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-socket-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.540210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540023    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4587d056-82b2-4204-9ce0-ab96aae93097-etc-tuned\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.540210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-cnibin\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.540210 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.540072    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:32.540210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540089    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-hostroot\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.540210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540052    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-hostroot\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.540234    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:33.040183393 +0000 UTC m=+2.026371452 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540247    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27k5k\" (UniqueName: \"kubernetes.io/projected/91276703-6b2d-40d1-8a70-1cf7d9b78570-kube-api-access-27k5k\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-cni-netd\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540320    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-var-lib-kubelet\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgvtq\" (UniqueName: \"kubernetes.io/projected/f76ffc5f-2c98-42fb-996f-a62e56716196-kube-api-access-mgvtq\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-cni-netd\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540405    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540445    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-systemd\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540481    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-system-cni-dir\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540510    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-ovn\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.540565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540561    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f911bedb-e01d-4f73-a947-89e2fd905115-dbus\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540587    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-system-cni-dir\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540587    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9kfj\" (UniqueName: \"kubernetes.io/projected/867afcc3-bfaa-457e-a67c-998900d3f853-kube-api-access-p9kfj\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540652    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysconfig\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540677    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-kubernetes\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-etc-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540726    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-env-overrides\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540743    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540774    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-registration-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-ovn\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/023334c1-c6b5-4cca-b99c-d7b1572a167d-iptables-alerter-script\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540851    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f911bedb-e01d-4f73-a947-89e2fd905115-dbus\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-systemd-units\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540882    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-etc-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540904    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-sys-fs\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540901    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.541190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540872    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540947    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-systemd-units\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.540956    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-cni-bin\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541000    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/786bcfda-4260-449e-b161-d57505ba1194-agent-certs\") pod \"konnectivity-agent-mr9tt\" (UID: \"786bcfda-4260-449e-b161-d57505ba1194\") " pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541023    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/786bcfda-4260-449e-b161-d57505ba1194-konnectivity-ca\") pod \"konnectivity-agent-mr9tt\" (UID: \"786bcfda-4260-449e-b161-d57505ba1194\") " pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541034    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-cni-bin\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-slash\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541068    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-log-socket\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541094    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541118    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-os-release\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541121    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-slash\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541140    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-os-release\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovnkube-config\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541171    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541188    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-lib-modules\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541211    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-host\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-log-socket\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xj6r9\" (UniqueName: \"kubernetes.io/projected/4587d056-82b2-4204-9ce0-ab96aae93097-kube-api-access-xj6r9\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541271    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-env-overrides\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541307    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-cnibin\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541301    2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541336    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-multus-certs\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91276703-6b2d-40d1-8a70-1cf7d9b78570-os-release\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/023334c1-c6b5-4cca-b99c-d7b1572a167d-iptables-alerter-script\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541362    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-etc-kubernetes\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-etc-kubernetes\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541393    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-cni-binary-copy\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541407    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-cnibin\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-var-lib-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovn-node-metrics-cert\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541464    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-os-release\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541476    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f911bedb-e01d-4f73-a947-89e2fd905115-kubelet-config\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541509    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-multus-certs\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541528    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f911bedb-e01d-4f73-a947-89e2fd905115-kubelet-config\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541540    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-device-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.542893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541565    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-var-lib-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-run-ovn-kubernetes\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541567    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-run-ovn-kubernetes\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz7jq\" (UniqueName: \"kubernetes.io/projected/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-kube-api-access-tz7jq\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541690    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bf9e6bf-545a-4ce9-b5ac-69d111087820-cni-binary-copy\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c66b2f39-3c48-45bc-9402-f5fbe55aecef-host\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/786bcfda-4260-449e-b161-d57505ba1194-konnectivity-ca\") pod \"konnectivity-agent-mr9tt\" (UID: \"786bcfda-4260-449e-b161-d57505ba1194\") " pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541737    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c66b2f39-3c48-45bc-9402-f5fbe55aecef-serviceca\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovnkube-config\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541760    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-node-log\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541784    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovnkube-script-lib\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541793    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c66b2f39-3c48-45bc-9402-f5fbe55aecef-host\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/867afcc3-bfaa-457e-a67c-998900d3f853-hosts-file\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-etc-selinux\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541861    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91276703-6b2d-40d1-8a70-1cf7d9b78570-cni-binary-copy\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541872    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-run\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541902    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-kubelet\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/867afcc3-bfaa-457e-a67c-998900d3f853-hosts-file\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.543931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541945    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/023334c1-c6b5-4cca-b99c-d7b1572a167d-host-slash\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541966    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-node-log\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.541986    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-sys\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-kubelet\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542012    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-conf-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542041    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-daemon-config\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-conf-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542120    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/023334c1-c6b5-4cca-b99c-d7b1572a167d-host-slash\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542153    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kp7s2\" (UniqueName: \"kubernetes.io/projected/c66b2f39-3c48-45bc-9402-f5fbe55aecef-kube-api-access-kp7s2\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542186    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fvjk\" (UniqueName: \"kubernetes.io/projected/023334c1-c6b5-4cca-b99c-d7b1572a167d-kube-api-access-8fvjk\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542215    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-run-netns\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542213    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c66b2f39-3c48-45bc-9402-f5fbe55aecef-serviceca\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542264    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-run-netns\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542481    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1bf9e6bf-545a-4ce9-b5ac-69d111087820-cni-binary-copy\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542501    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-cni-bin\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542642    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84l6l\" (UniqueName: \"kubernetes.io/projected/7eef7004-beaf-4b1e-824f-ffdaad438624-kube-api-access-84l6l\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542649    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovnkube-script-lib\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-cni-multus\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.544656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542716    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-cni-bin\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542751    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-kubelet\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542773    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-daemon-config\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-var-lib-cni-multus\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542854    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-host-kubelet\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542862    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867afcc3-bfaa-457e-a67c-998900d3f853-tmp-dir\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.542911    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.542959    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-modprobe-d\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:32.542992    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:33.042965195 +0000 UTC m=+2.029153239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543018    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysctl-d\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543034    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543058    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-cni-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543077    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-netns\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543091    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4587d056-82b2-4204-9ce0-ab96aae93097-tmp\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543106    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-system-cni-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543128    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-k8s-cni-cncf-io\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543145    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-multus-cni-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543158    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-netns\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-systemd\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543192    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/867afcc3-bfaa-457e-a67c-998900d3f853-tmp-dir\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543215    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysctl-conf\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543201    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-host-run-k8s-cni-cncf-io\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543242    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1bf9e6bf-545a-4ce9-b5ac-69d111087820-system-cni-dir\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-openvswitch\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.543301    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-run-systemd\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.544558    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-ovn-node-metrics-cert\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.545837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.544665    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/786bcfda-4260-449e-b161-d57505ba1194-agent-certs\") pod \"konnectivity-agent-mr9tt\" (UID: \"786bcfda-4260-449e-b161-d57505ba1194\") " pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:32.550055 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.549971    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27k5k\" (UniqueName: \"kubernetes.io/projected/91276703-6b2d-40d1-8a70-1cf7d9b78570-kube-api-access-27k5k\") pod \"multus-additional-cni-plugins-xsk6j\" (UID: \"91276703-6b2d-40d1-8a70-1cf7d9b78570\") " pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:32.550055 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.549977    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-twmr6\" (UniqueName: \"kubernetes.io/projected/1bf9e6bf-545a-4ce9-b5ac-69d111087820-kube-api-access-twmr6\") pod \"multus-cwgjq\" (UID: \"1bf9e6bf-545a-4ce9-b5ac-69d111087820\") " pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:32.550316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.550294    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9kfj\" (UniqueName: \"kubernetes.io/projected/867afcc3-bfaa-457e-a67c-998900d3f853-kube-api-access-p9kfj\") pod \"node-resolver-8z6tc\" (UID: \"867afcc3-bfaa-457e-a67c-998900d3f853\") " pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:32.550815 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.550794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgvtq\" (UniqueName: \"kubernetes.io/projected/f76ffc5f-2c98-42fb-996f-a62e56716196-kube-api-access-mgvtq\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:32.551284 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.551258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kp7s2\" (UniqueName: \"kubernetes.io/projected/c66b2f39-3c48-45bc-9402-f5fbe55aecef-kube-api-access-kp7s2\") pod \"node-ca-qwlrm\" (UID: \"c66b2f39-3c48-45bc-9402-f5fbe55aecef\") " pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:32.551477 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.551456    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz7jq\" (UniqueName: \"kubernetes.io/projected/d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40-kube-api-access-tz7jq\") pod \"ovnkube-node-6dxmp\" (UID: \"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40\") " pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:32.552104 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:32.552084    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fvjk\" (UniqueName: \"kubernetes.io/projected/023334c1-c6b5-4cca-b99c-d7b1572a167d-kube-api-access-8fvjk\") pod \"iptables-alerter-9p9jl\" (UID: \"023334c1-c6b5-4cca-b99c-d7b1572a167d\") " pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:32.625878 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:32.625838    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ddc1535de6c0eb54f69fb18e4b2deac.slice/crio-315d846e06ece2b22048487698c37f995677503d80dcc4b800ba1b8665ea186e WatchSource:0}: Error finding container 315d846e06ece2b22048487698c37f995677503d80dcc4b800ba1b8665ea186e: Status 404 returned error can't find the container with id 315d846e06ece2b22048487698c37f995677503d80dcc4b800ba1b8665ea186e
Jul 13 21:11:32.626095 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:32.626078    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2be0832c80a981e87f324caa4fb613b.slice/crio-ab913c6e53a6b7be80e28e543f92a407affdec38f7bf4625291eeb641addeef3 WatchSource:0}: Error finding container ab913c6e53a6b7be80e28e543f92a407affdec38f7bf4625291eeb641addeef3: Status 404 returned error can't find the container with id ab913c6e53a6b7be80e28e543f92a407affdec38f7bf4625291eeb641addeef3
Jul 13 21:11:34.104959 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.104942    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:11:34.117590 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysconfig\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117676 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117596    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-kubernetes\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117676 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117676 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-registration-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117676 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117648    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-sys-fs\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117693    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-lib-modules\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117705    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-kubernetes\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117718    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-registration-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117712    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysconfig\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117715    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-host\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117770    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-sys-fs\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117795    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xj6r9\" (UniqueName: \"kubernetes.io/projected/4587d056-82b2-4204-9ce0-ab96aae93097-kube-api-access-xj6r9\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-device-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117823    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-host\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117849    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-etc-selinux\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.117869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117856    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-lib-modules\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117873    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-run\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117891    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-device-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-sys\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117942    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-etc-selinux\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117950    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84l6l\" (UniqueName: \"kubernetes.io/projected/7eef7004-beaf-4b1e-824f-ffdaad438624-kube-api-access-84l6l\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117963    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-run\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.117972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-sys\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118086    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-modprobe-d\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118113    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysctl-d\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118193    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-modprobe-d\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118200    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4587d056-82b2-4204-9ce0-ab96aae93097-tmp\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysctl-conf\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118285    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysctl-d\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-socket-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118333    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4587d056-82b2-4204-9ce0-ab96aae93097-etc-tuned\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.118367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118367    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-var-lib-kubelet\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.119089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118395    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-sysctl-conf\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.119089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118396    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-systemd\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.119089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118467    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7eef7004-beaf-4b1e-824f-ffdaad438624-socket-dir\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.119089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-var-lib-kubelet\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.119089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.118519    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/4587d056-82b2-4204-9ce0-ab96aae93097-etc-systemd\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.120479 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.120452    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/4587d056-82b2-4204-9ce0-ab96aae93097-etc-tuned\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.120576 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.120560    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4587d056-82b2-4204-9ce0-ab96aae93097-tmp\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.125801 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.125785    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:34.125888 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.125804    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:34.125888 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.125814    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:34.125888 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.125870    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:11:34.625857442 +0000 UTC m=+2.138607162 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:34.127829 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.127810    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84l6l\" (UniqueName: \"kubernetes.io/projected/7eef7004-beaf-4b1e-824f-ffdaad438624-kube-api-access-84l6l\") pod \"aws-ebs-csi-driver-node-2tztl\" (UID: \"7eef7004-beaf-4b1e-824f-ffdaad438624\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.130120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.128088    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xj6r9\" (UniqueName: \"kubernetes.io/projected/4587d056-82b2-4204-9ce0-ab96aae93097-kube-api-access-xj6r9\") pod \"tuned-cgp69\" (UID: \"4587d056-82b2-4204-9ce0-ab96aae93097\") " pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.221098 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.221048    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xsk6j"
Jul 13 21:11:34.229008 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.228983    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91276703_6b2d_40d1_8a70_1cf7d9b78570.slice/crio-98ed2805798b0841e682b9560e343d5a98a78defb3c502c531df52661eb5df41 WatchSource:0}: Error finding container 98ed2805798b0841e682b9560e343d5a98a78defb3c502c531df52661eb5df41: Status 404 returned error can't find the container with id 98ed2805798b0841e682b9560e343d5a98a78defb3c502c531df52661eb5df41
Jul 13 21:11:34.230133 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.230113    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl"
Jul 13 21:11:34.237099 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.237079    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eef7004_beaf_4b1e_824f_ffdaad438624.slice/crio-59fe6fa68d40cfce3f808e0f220c1a6e0a02230f1f17a8d0986a26fb108efe03 WatchSource:0}: Error finding container 59fe6fa68d40cfce3f808e0f220c1a6e0a02230f1f17a8d0986a26fb108efe03: Status 404 returned error can't find the container with id 59fe6fa68d40cfce3f808e0f220c1a6e0a02230f1f17a8d0986a26fb108efe03
Jul 13 21:11:34.247342 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.247322    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-9p9jl"
Jul 13 21:11:34.252085 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.252062    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:11:34.254085 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.254066    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod023334c1_c6b5_4cca_b99c_d7b1572a167d.slice/crio-c22a303548cf34f19bdc213136902dbf42631d0fccd205b780aad3f96a5e01ab WatchSource:0}: Error finding container c22a303548cf34f19bdc213136902dbf42631d0fccd205b780aad3f96a5e01ab: Status 404 returned error can't find the container with id c22a303548cf34f19bdc213136902dbf42631d0fccd205b780aad3f96a5e01ab
Jul 13 21:11:34.259272 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.259251    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8d4ba3e_6aa4_4d06_9ebd_11c2ba351c40.slice/crio-d437d8c9d5ee99b86198014aab7f18b6e859de6ecd944050e71d085a62c53f7d WatchSource:0}: Error finding container d437d8c9d5ee99b86198014aab7f18b6e859de6ecd944050e71d085a62c53f7d: Status 404 returned error can't find the container with id d437d8c9d5ee99b86198014aab7f18b6e859de6ecd944050e71d085a62c53f7d
Jul 13 21:11:34.268087 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.268068    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:34.274848 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.274826    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786bcfda_4260_449e_b161_d57505ba1194.slice/crio-78b44eb87ee8216d372c0d9653a181280d8e3741aafbaa7033c4e78101e6cc16 WatchSource:0}: Error finding container 78b44eb87ee8216d372c0d9653a181280d8e3741aafbaa7033c4e78101e6cc16: Status 404 returned error can't find the container with id 78b44eb87ee8216d372c0d9653a181280d8e3741aafbaa7033c4e78101e6cc16
Jul 13 21:11:34.285907 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.285889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8z6tc"
Jul 13 21:11:34.292555 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.292533    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod867afcc3_bfaa_457e_a67c_998900d3f853.slice/crio-31f35400cd6366ceb0f049f2cd9abed9f2b89ca37c9925c990199f250137da5b WatchSource:0}: Error finding container 31f35400cd6366ceb0f049f2cd9abed9f2b89ca37c9925c990199f250137da5b: Status 404 returned error can't find the container with id 31f35400cd6366ceb0f049f2cd9abed9f2b89ca37c9925c990199f250137da5b
Jul 13 21:11:34.304193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.304175    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-qwlrm"
Jul 13 21:11:34.310575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.310555    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-cwgjq"
Jul 13 21:11:34.310774 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.310752    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc66b2f39_3c48_45bc_9402_f5fbe55aecef.slice/crio-72e0c874aed97e02ce9a9b6f784cd41b5ac83a000eccefdab534a645dacb4f3e WatchSource:0}: Error finding container 72e0c874aed97e02ce9a9b6f784cd41b5ac83a000eccefdab534a645dacb4f3e: Status 404 returned error can't find the container with id 72e0c874aed97e02ce9a9b6f784cd41b5ac83a000eccefdab534a645dacb4f3e
Jul 13 21:11:34.314780 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.314759    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-cgp69"
Jul 13 21:11:34.319130 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.319103    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf9e6bf_545a_4ce9_b5ac_69d111087820.slice/crio-3b8db6e6262823ffa4140dc22327238fa2f13f84025f1169f1b113b37c2a0509 WatchSource:0}: Error finding container 3b8db6e6262823ffa4140dc22327238fa2f13f84025f1169f1b113b37c2a0509: Status 404 returned error can't find the container with id 3b8db6e6262823ffa4140dc22327238fa2f13f84025f1169f1b113b37c2a0509
Jul 13 21:11:34.322697 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:11:34.322679    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4587d056_82b2_4204_9ce0_ab96aae93097.slice/crio-09f2c7cb29d52a180d797dad01a0e82f9472bbddd8d52f68f4538152aeab798a WatchSource:0}: Error finding container 09f2c7cb29d52a180d797dad01a0e82f9472bbddd8d52f68f4538152aeab798a: Status 404 returned error can't find the container with id 09f2c7cb29d52a180d797dad01a0e82f9472bbddd8d52f68f4538152aeab798a
Jul 13 21:11:34.520682 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.520603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:34.520682 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.520659    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:34.520895 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.520770    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:34.520895 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.520784    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:34.520895 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.520852    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:35.520831843 +0000 UTC m=+3.033581581 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:34.520895 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.520873    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:35.520863286 +0000 UTC m=+3.033613006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:34.722408 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.722368    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:34.722641 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.722619    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:34.722718 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.722652    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:34.722718 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.722668    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:34.722839 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:34.722735    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:11:35.722714029 +0000 UTC m=+3.235463751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:34.942812 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.942697    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:06:33 +0000 UTC" deadline="2028-04-01 00:46:08.61457004 +0000 UTC"
Jul 13 21:11:34.942812 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:34.942763    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15051h34m33.671811003s"
Jul 13 21:11:35.071753 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.070912    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:35.071753 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.071065    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:35.071753 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.071481    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:35.071753 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.071584    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:35.071753 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.071648    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:35.071753 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.071714    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:35.084038 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.083810    2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:35.098003 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.097850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl" event={"ID":"7eef7004-beaf-4b1e-824f-ffdaad438624","Type":"ContainerStarted","Data":"59fe6fa68d40cfce3f808e0f220c1a6e0a02230f1f17a8d0986a26fb108efe03"}
Jul 13 21:11:35.104911 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.104868    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal" event={"ID":"0ddc1535de6c0eb54f69fb18e4b2deac","Type":"ContainerStarted","Data":"315d846e06ece2b22048487698c37f995677503d80dcc4b800ba1b8665ea186e"}
Jul 13 21:11:35.120069 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.120026    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal" event={"ID":"a2be0832c80a981e87f324caa4fb613b","Type":"ContainerStarted","Data":"ab913c6e53a6b7be80e28e543f92a407affdec38f7bf4625291eeb641addeef3"}
Jul 13 21:11:35.137309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.137234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cwgjq" event={"ID":"1bf9e6bf-545a-4ce9-b5ac-69d111087820","Type":"ContainerStarted","Data":"3b8db6e6262823ffa4140dc22327238fa2f13f84025f1169f1b113b37c2a0509"}
Jul 13 21:11:35.145413 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.143342    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwlrm" event={"ID":"c66b2f39-3c48-45bc-9402-f5fbe55aecef","Type":"ContainerStarted","Data":"72e0c874aed97e02ce9a9b6f784cd41b5ac83a000eccefdab534a645dacb4f3e"}
Jul 13 21:11:35.152998 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.152971    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerStarted","Data":"98ed2805798b0841e682b9560e343d5a98a78defb3c502c531df52661eb5df41"}
Jul 13 21:11:35.160819 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.160796    2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:35.168737 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.168713    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cgp69" event={"ID":"4587d056-82b2-4204-9ce0-ab96aae93097","Type":"ContainerStarted","Data":"09f2c7cb29d52a180d797dad01a0e82f9472bbddd8d52f68f4538152aeab798a"}
Jul 13 21:11:35.191746 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.191719    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8z6tc" event={"ID":"867afcc3-bfaa-457e-a67c-998900d3f853","Type":"ContainerStarted","Data":"31f35400cd6366ceb0f049f2cd9abed9f2b89ca37c9925c990199f250137da5b"}
Jul 13 21:11:35.204965 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.204858    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mr9tt" event={"ID":"786bcfda-4260-449e-b161-d57505ba1194","Type":"ContainerStarted","Data":"78b44eb87ee8216d372c0d9653a181280d8e3741aafbaa7033c4e78101e6cc16"}
Jul 13 21:11:35.216568 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.216543    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"d437d8c9d5ee99b86198014aab7f18b6e859de6ecd944050e71d085a62c53f7d"}
Jul 13 21:11:35.235200 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.234933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9p9jl" event={"ID":"023334c1-c6b5-4cca-b99c-d7b1572a167d","Type":"ContainerStarted","Data":"c22a303548cf34f19bdc213136902dbf42631d0fccd205b780aad3f96a5e01ab"}
Jul 13 21:11:35.528616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.528522    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:35.528616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.528589    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:35.528822 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.528733    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:35.528822 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.528793    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:37.528774919 +0000 UTC m=+5.041524642 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:35.529235 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.529210    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:35.529347 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.529267    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:37.529251669 +0000 UTC m=+5.042001409 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:35.553187 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.553159    2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 13 21:11:35.730125 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.730092    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:35.730299 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.730281    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:35.730367 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.730302    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:35.730367 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.730316    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:35.730466 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:35.730370    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:11:37.730353318 +0000 UTC m=+5.243103045 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:35.944006 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.943646    2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 21:06:33 +0000 UTC" deadline="2028-02-07 11:38:34.378187918 +0000 UTC"
Jul 13 21:11:35.944006 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:35.943682    2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13766h26m58.434511245s"
Jul 13 21:11:37.071435 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:37.071149    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:37.071871 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.071516    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:37.071871 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:37.071206    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:37.071871 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.071609    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:37.071871 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:37.071172    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:37.071871 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.071701    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:37.547198 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:37.547108    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:37.547198 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:37.547179    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:37.547405 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.547301    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:37.547405 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.547362    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:41.547341911 +0000 UTC m=+9.060091659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:37.547790 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.547765    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:37.547905 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.547822    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:41.547805797 +0000 UTC m=+9.060555517 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:37.749689 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:37.749653    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:37.749874 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.749837    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:37.749874 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.749856    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:37.749874 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.749868    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:37.750039 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:37.749923    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:11:41.749905565 +0000 UTC m=+9.262655289 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:39.070941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:39.070746    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:39.070941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:39.070781    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:39.070941 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:39.070886    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:39.071537 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:39.071016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:39.071537 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:39.071153    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:39.071537 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:39.071237    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:41.070418 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:41.070171    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:41.070418 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:41.070216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:41.070418 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:41.070184    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:41.070418 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.070311    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:41.071035 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.070421    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:41.071035 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.070529    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:41.579445 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:41.579271    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:41.579445 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:41.579343    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:41.579705 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.579452    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:41.579705 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.579520    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:49.57950105 +0000 UTC m=+17.092250783 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:41.579705 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.579518    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:41.579705 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.579586    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:11:49.579567346 +0000 UTC m=+17.092317091 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:41.781251 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:41.781209    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:41.781934 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.781435    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:41.781934 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.781468    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:41.781934 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.781483    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:41.781934 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:41.781560    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:11:49.781540888 +0000 UTC m=+17.294290611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:43.071416 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:43.071385    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:43.071848 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:43.071508    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:43.071913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:43.071878    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:43.071988 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:43.071968    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:43.072042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:43.072025    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:43.072128 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:43.072110    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:45.071115 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:45.071080    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:45.071115 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:45.071109    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:45.071599 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:45.071080    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:45.071599 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:45.071212    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:45.071599 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:45.071322    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:45.071599 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:45.071432    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:47.070383 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:47.070342    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:47.070850 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:47.070342    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:47.070850 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:47.070465    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:47.070850 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:47.070479    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:47.070850 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:47.070552    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:47.070850 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:47.070660    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:49.070562 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:49.070528    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:49.070562 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:49.070547    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:49.071048 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.070658    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:49.071132 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.071099    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:49.071261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:49.071243    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:49.071387 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.071367    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:49.644315 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:49.644278    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:49.644473 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:49.644341    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:49.644473 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.644436    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:49.644473 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.644448    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:49.644596 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.644502    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:05.644482784 +0000 UTC m=+33.157232517 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:11:49.644596 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.644523    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:05.644512589 +0000 UTC m=+33.157262310 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:11:49.845005 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:49.844969    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:49.845188 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.845130    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:11:49.845188 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.845150    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:11:49.845188 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.845158    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:49.845368 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:49.845216    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:12:05.845195153 +0000 UTC m=+33.357944874 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:11:51.070160 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:51.070118    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:51.070718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:51.070164    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:51.070718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:51.070172    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:51.070718 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:51.070267    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:51.070718 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:51.070351    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:51.070718 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:51.070426    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:53.071267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:53.071238    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:53.071692 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:53.071353    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:53.071692 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:53.071377    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:53.071692 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:53.071349    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:53.071692 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:53.071469    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:53.071692 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:53.071593    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:54.279316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:54.279292    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal" event={"ID":"0ddc1535de6c0eb54f69fb18e4b2deac","Type":"ContainerStarted","Data":"20639452c9bea1acecf0baf351413034f171712069617a3a090baa080ed4e90f"}
Jul 13 21:11:54.280629 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:54.280607    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-cgp69" event={"ID":"4587d056-82b2-4204-9ce0-ab96aae93097","Type":"ContainerStarted","Data":"341a20408ec864ac467009201fcf999bfe033fe5a2d066ad04df1f2c56579afc"}
Jul 13 21:11:54.292041 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:54.292003    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-147.ec2.internal" podStartSLOduration=21.291990487 podStartE2EDuration="21.291990487s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:54.291772761 +0000 UTC m=+21.804522502" watchObservedRunningTime="2026-07-13 21:11:54.291990487 +0000 UTC m=+21.804740227"
Jul 13 21:11:55.070824 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.070438    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:55.070824 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.070488    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:55.071139 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:55.070841    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:55.071139 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:55.070911    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:55.071139 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.070439    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:55.071139 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:55.070994    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:55.284425 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.284392    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8z6tc" event={"ID":"867afcc3-bfaa-457e-a67c-998900d3f853","Type":"ContainerStarted","Data":"3e28a0820fd26aefc6d99d392a4850b933c0e3fd62a3f93507bbaf080995feb0"}
Jul 13 21:11:55.285910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.285885    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mr9tt" event={"ID":"786bcfda-4260-449e-b161-d57505ba1194","Type":"ContainerStarted","Data":"4b842390bddacc5f7b15b8295870bdaf0c9ccb378596a640118071f73475ec43"}
Jul 13 21:11:55.288532 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.288516    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:11:55.288838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.288820    2564 generic.go:356] "Generic (PLEG): container finished" podID="d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40" containerID="db5f055d5a135e78846686835ee10a76f567565885675f49ace07d1f0d680f6d" exitCode=1
Jul 13 21:11:55.288903 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.288887    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"f7e102af4b547b583f9bfd60406d116880ed1b0718b38359b02a7b1626187932"}
Jul 13 21:11:55.288954 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.288940    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"c3c621a6af1fe71cae20143d1222f5a9b742b93225683a63ed5c8c9982c50af6"}
Jul 13 21:11:55.288994 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.288983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"03aea0981e33e25f0c9161b9b0696377983d3dc1554e66fe3966ecba6b041b05"}
Jul 13 21:11:55.289024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.289012    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"29de17666a3e5e07ff42c2ed409d30889c8b93486f902a4eaf4935ee8c077a26"}
Jul 13 21:11:55.289074 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.289065    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"8cc7b7e247831b00848987162874c2f6a345d701835002b332f975b24faec3b4"}
Jul 13 21:11:55.289113 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.289101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerDied","Data":"db5f055d5a135e78846686835ee10a76f567565885675f49ace07d1f0d680f6d"}
Jul 13 21:11:55.290268 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.290242    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl" event={"ID":"7eef7004-beaf-4b1e-824f-ffdaad438624","Type":"ContainerStarted","Data":"4efa9de281a3b6923102652783ba6f987946d9628f57913a1daaeddff0276b49"}
Jul 13 21:11:55.291708 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.291686    2564 generic.go:356] "Generic (PLEG): container finished" podID="a2be0832c80a981e87f324caa4fb613b" containerID="1486cdd128e8b725ce60933ff211ef947832ebc6aa7630334527e6fad94f8725" exitCode=0
Jul 13 21:11:55.291792 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.291746    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal" event={"ID":"a2be0832c80a981e87f324caa4fb613b","Type":"ContainerDied","Data":"1486cdd128e8b725ce60933ff211ef947832ebc6aa7630334527e6fad94f8725"}
Jul 13 21:11:55.293275 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.293260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-cwgjq" event={"ID":"1bf9e6bf-545a-4ce9-b5ac-69d111087820","Type":"ContainerStarted","Data":"a4a0cb7215ddea234d357e9dbe6dd46cd1794136dd17b2d7db7976cd0a1e788c"}
Jul 13 21:11:55.294574 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.294552    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-qwlrm" event={"ID":"c66b2f39-3c48-45bc-9402-f5fbe55aecef","Type":"ContainerStarted","Data":"d7753a25ce528a033fe8acff7554964558628c804e344e13e13549b4b0d072fc"}
Jul 13 21:11:55.295983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.295962    2564 generic.go:356] "Generic (PLEG): container finished" podID="91276703-6b2d-40d1-8a70-1cf7d9b78570" containerID="52f48e3db4982190f27d2ff7e30695d6e8bf92dd6f9cfe976cdb5c26b69a666b" exitCode=0
Jul 13 21:11:55.296081 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.296043    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerDied","Data":"52f48e3db4982190f27d2ff7e30695d6e8bf92dd6f9cfe976cdb5c26b69a666b"}
Jul 13 21:11:55.301065 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.301018    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8z6tc" podStartSLOduration=2.63141085 podStartE2EDuration="22.301008093s" totalImagesPullingTime="19.669597243s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:55.300804548 +0000 UTC m=+22.813554287" watchObservedRunningTime="2026-07-13 21:11:55.301008093 +0000 UTC m=+22.813757836"
Jul 13 21:11:55.318659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.318621    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mr9tt" podStartSLOduration=10.529889838 podStartE2EDuration="22.31860788s" totalImagesPullingTime="11.788718042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:55.318133649 +0000 UTC m=+22.830883391" watchObservedRunningTime="2026-07-13 21:11:55.31860788 +0000 UTC m=+22.831357621"
Jul 13 21:11:55.345942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.345902    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-cwgjq" podStartSLOduration=2.209105085 podStartE2EDuration="22.345887383s" totalImagesPullingTime="20.136782298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:55.345743436 +0000 UTC m=+22.858493176" watchObservedRunningTime="2026-07-13 21:11:55.345887383 +0000 UTC m=+22.858637123"
Jul 13 21:11:55.365336 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.365292    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-cgp69" podStartSLOduration=2.583128615 podStartE2EDuration="22.365279523s" totalImagesPullingTime="19.782150908s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:55.36479274 +0000 UTC m=+22.877542481" watchObservedRunningTime="2026-07-13 21:11:55.365279523 +0000 UTC m=+22.878029267"
Jul 13 21:11:55.379422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:55.379385    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-qwlrm" podStartSLOduration=2.58635342 podStartE2EDuration="22.379372502s" totalImagesPullingTime="19.793019082s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:55.379342842 +0000 UTC m=+22.892092584" watchObservedRunningTime="2026-07-13 21:11:55.379372502 +0000 UTC m=+22.892122242"
Jul 13 21:11:56.301229 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:56.301179    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-9p9jl" event={"ID":"023334c1-c6b5-4cca-b99c-d7b1572a167d","Type":"ContainerStarted","Data":"d3780ae9af2f660e3afadf99af1aa2a78e49152229576c9e50350a7f338c94e6"}
Jul 13 21:11:56.304746 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:56.304715    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal" event={"ID":"a2be0832c80a981e87f324caa4fb613b","Type":"ContainerStarted","Data":"9fa511ea0a4dba22cd68d53e68f360b3f735c22caaf9aa33d72a47332d29cefb"}
Jul 13 21:11:56.332800 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:56.332736    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-147.ec2.internal" podStartSLOduration=23.332715641 podStartE2EDuration="23.332715641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:56.332380768 +0000 UTC m=+23.845130513" watchObservedRunningTime="2026-07-13 21:11:56.332715641 +0000 UTC m=+23.845465383"
Jul 13 21:11:56.332991 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:56.332936    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-9p9jl" podStartSLOduration=3.615376492 podStartE2EDuration="23.33293034s" totalImagesPullingTime="19.717553848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:11:56.317073586 +0000 UTC m=+23.829823328" watchObservedRunningTime="2026-07-13 21:11:56.33293034 +0000 UTC m=+23.845680097"
Jul 13 21:11:57.070773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.070735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:57.070773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.070748    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:57.071025 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.070735    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:57.071025 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:57.070853    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:57.071025 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:57.070956    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:57.071215 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:57.071047    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:11:57.128939 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.128783    2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 13 21:11:57.309935 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.309866    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:11:57.310773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.310677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"e77d0052a623b7123221afb605078c68e6109490622118491bd4c020042f85a9"}
Jul 13 21:11:57.313327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.313299    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl" event={"ID":"7eef7004-beaf-4b1e-824f-ffdaad438624","Type":"ContainerStarted","Data":"fbd9e6e8bfa2e28da72fa663157f8e4f160c1e5cd4da4acf5919c64e23af0798"}
Jul 13 21:11:57.486770 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.486738    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:57.487401 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.487373    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:57.981088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.980943    2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T21:11:57.128801246Z","UUID":"9852e21b-e140-4632-9868-a505bfb84630","Handler":null,"Name":"","Endpoint":""}
Jul 13 21:11:57.982728 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.982702    2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 13 21:11:57.982850 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:57.982747    2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 13 21:11:58.316159 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:58.316079    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:58.316712 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:58.316694    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mr9tt"
Jul 13 21:11:59.070703 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:59.070672    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:11:59.070703 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:59.070692    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:11:59.070949 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:59.070790    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:11:59.070949 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:59.070867    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:11:59.070949 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:11:59.070918    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:11:59.071134 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:11:59.071029    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:00.324217 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.324035    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:12:00.325012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.324551    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"49fb809c054a98233fed4472a37f69fceec6d96e191156f236f1415920cc09c7"}
Jul 13 21:12:00.325012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.324793    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:12:00.325012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.324994    2564 scope.go:117] "RemoveContainer" containerID="db5f055d5a135e78846686835ee10a76f567565885675f49ace07d1f0d680f6d"
Jul 13 21:12:00.326873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.326850    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl" event={"ID":"7eef7004-beaf-4b1e-824f-ffdaad438624","Type":"ContainerStarted","Data":"0d91ce2b391c4826f6b8d57e402c601a695ec6e92c362e9386def7b893395cb0"}
Jul 13 21:12:00.328801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.328777    2564 generic.go:356] "Generic (PLEG): container finished" podID="91276703-6b2d-40d1-8a70-1cf7d9b78570" containerID="56b910e10862ab22f499c467a524f4f35756cf7970863980569dba3c8e5007e1" exitCode=0
Jul 13 21:12:00.328905 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.328865    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerDied","Data":"56b910e10862ab22f499c467a524f4f35756cf7970863980569dba3c8e5007e1"}
Jul 13 21:12:00.342460 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:00.342437    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:12:01.071249 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.070997    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:01.071450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.071108    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:01.071450 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:01.071329    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:01.071450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.071120    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:01.071450 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:01.071392    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:01.071650 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:01.071458    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:01.334333 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.334263    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:12:01.334748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.334658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" event={"ID":"d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40","Type":"ContainerStarted","Data":"35195bdc17eb1786202f80e97a05e45a11a67d17c408fe04cc604410e273ef37"}
Jul 13 21:12:01.334903 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.334872    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:12:01.335034 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.334950    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:12:01.349908 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.349886    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:12:01.382756 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.382702    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2tztl" podStartSLOduration=3.777881661 podStartE2EDuration="28.382687362s" totalImagesPullingTime="24.604805701s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:00.415949172 +0000 UTC m=+27.928698913" watchObservedRunningTime="2026-07-13 21:12:01.382687362 +0000 UTC m=+28.895437103"
Jul 13 21:12:01.382983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:01.382958    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp" podStartSLOduration=8.495136374 podStartE2EDuration="28.382953246s" totalImagesPullingTime="19.887816872s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:01.382469464 +0000 UTC m=+28.895219204" watchObservedRunningTime="2026-07-13 21:12:01.382953246 +0000 UTC m=+28.895702987"
Jul 13 21:12:02.338776 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:02.338745    2564 generic.go:356] "Generic (PLEG): container finished" podID="91276703-6b2d-40d1-8a70-1cf7d9b78570" containerID="83bbdb4c74b2472359d586b1b38626e126632b1d4c2ce1d8f498c840420fbcc7" exitCode=0
Jul 13 21:12:02.339199 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:02.338841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerDied","Data":"83bbdb4c74b2472359d586b1b38626e126632b1d4c2ce1d8f498c840420fbcc7"}
Jul 13 21:12:03.071484 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:03.071450    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:03.071680 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:03.071538    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:03.071680 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:03.071539    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:03.071680 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:03.071667    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:03.071855 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:03.071703    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:03.071855 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:03.071779    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:04.345390 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:04.345356    2564 generic.go:356] "Generic (PLEG): container finished" podID="91276703-6b2d-40d1-8a70-1cf7d9b78570" containerID="0c34373b032ecd29c31b0fe183653574d1eb8adb12c9446e6ccf3d4d7f6fa1e6" exitCode=0
Jul 13 21:12:04.345759 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:04.345427    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerDied","Data":"0c34373b032ecd29c31b0fe183653574d1eb8adb12c9446e6ccf3d4d7f6fa1e6"}
Jul 13 21:12:05.070415 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:05.070379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:05.070603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:05.070379    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:05.070603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:05.070515    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:05.070603 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.070509    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:05.070764 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.070615    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:05.070764 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.070697    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:05.666812 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:05.666781    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:05.667287 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:05.666832    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:05.667287 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.666945    2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 13 21:12:05.667287 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.666955    2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:12:05.667287 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.667030    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret podName:f911bedb-e01d-4f73-a947-89e2fd905115 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:37.667008656 +0000 UTC m=+65.179758375 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret") pod "global-pull-secret-syncer-czhjv" (UID: "f911bedb-e01d-4f73-a947-89e2fd905115") : object "kube-system"/"original-pull-secret" not registered
Jul 13 21:12:05.667287 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.667050    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs podName:f76ffc5f-2c98-42fb-996f-a62e56716196 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:37.667040909 +0000 UTC m=+65.179790628 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs") pod "network-metrics-daemon-q7js4" (UID: "f76ffc5f-2c98-42fb-996f-a62e56716196") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 13 21:12:05.868708 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:05.868676    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:05.868904 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.868883    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 13 21:12:05.868950 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.868912    2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 13 21:12:05.868983 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.868975    2564 projected.go:196] Error preparing data for projected volume kube-api-access-m4mwr for pod openshift-network-diagnostics/network-check-target-qs78m: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:12:05.869065 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:05.869032    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr podName:bd10968a-4ce3-40e8-9e06-26b1bca018dc nodeName:}" failed. No retries permitted until 2026-07-13 21:12:37.869013469 +0000 UTC m=+65.381763210 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m4mwr" (UniqueName: "kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr") pod "network-check-target-qs78m" (UID: "bd10968a-4ce3-40e8-9e06-26b1bca018dc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 13 21:12:07.070349 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:07.070312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:07.070801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:07.070312    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:07.070801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:07.070321    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:07.070801 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:07.070464    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:07.070801 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:07.070553    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:07.070801 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:07.070641    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:09.071188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:09.071098    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:09.071188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:09.071136    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:09.071613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:09.071098    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:09.071613 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:09.071226    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:09.071613 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:09.071265    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:09.071613 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:09.071346    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:09.363417 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:09.363332    2564 generic.go:356] "Generic (PLEG): container finished" podID="91276703-6b2d-40d1-8a70-1cf7d9b78570" containerID="1848b44654c6b62fc43576d091cc504bb20b5ce22ad51892f9a5267236ec79a2" exitCode=0
Jul 13 21:12:09.363417 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:09.363396    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerDied","Data":"1848b44654c6b62fc43576d091cc504bb20b5ce22ad51892f9a5267236ec79a2"}
Jul 13 21:12:10.371551 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:10.371515    2564 generic.go:356] "Generic (PLEG): container finished" podID="91276703-6b2d-40d1-8a70-1cf7d9b78570" containerID="8150a5a024b0b8a5ac5af128d871d2fe49414087ab7d45ff4cb1945d154c9104" exitCode=0
Jul 13 21:12:10.371551 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:10.371550    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerDied","Data":"8150a5a024b0b8a5ac5af128d871d2fe49414087ab7d45ff4cb1945d154c9104"}
Jul 13 21:12:11.070222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:11.070193    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:11.070222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:11.070203    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:11.070222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:11.070214    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:11.070464 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:11.070285    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:11.070464 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:11.070353    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:11.070464 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:11.070417    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:11.376773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:11.376700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" event={"ID":"91276703-6b2d-40d1-8a70-1cf7d9b78570","Type":"ContainerStarted","Data":"8e671dc0ef506ee71b8f56830b3f62db7762c6edb149b33636aca40067be051d"}
Jul 13 21:12:11.399540 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:11.399501    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xsk6j" podStartSLOduration=15.371766615 podStartE2EDuration="38.399487426s" totalImagesPullingTime="23.027720811s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:11.3979899 +0000 UTC m=+38.910739642" watchObservedRunningTime="2026-07-13 21:12:11.399487426 +0000 UTC m=+38.912237167"
Jul 13 21:12:13.072474 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:13.072426    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:13.073019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:13.072520    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:13.073019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:13.072564    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:13.073019 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:13.072572    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:13.073019 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:13.072682    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:13.073019 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:13.072804    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:14.150811 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:14.150611    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-czhjv"]
Jul 13 21:12:14.151286 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:14.150886    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:14.151286 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:14.150986    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:14.154158 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:14.154133    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qs78m"]
Jul 13 21:12:14.154277 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:14.154251    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:14.154377 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:14.154354    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:14.154814 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:14.154781    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7js4"]
Jul 13 21:12:14.154903 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:14.154874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:14.155002 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:14.154981    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:16.070995 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:16.070945    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:16.070995 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:16.070975    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:16.070995 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:16.070989    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:16.071509 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:16.071086    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:16.071509 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:16.071193    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:16.071509 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:16.071264    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:18.071096 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.071040    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:18.071096 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.071083    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:18.071096 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.071093    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:18.071685 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:18.071175    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qs78m" podUID="bd10968a-4ce3-40e8-9e06-26b1bca018dc"
Jul 13 21:12:18.071685 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:18.071282    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-czhjv" podUID="f911bedb-e01d-4f73-a947-89e2fd905115"
Jul 13 21:12:18.071685 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:18.071347    2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7js4" podUID="f76ffc5f-2c98-42fb-996f-a62e56716196"
Jul 13 21:12:18.253230 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.253141    2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-147.ec2.internal" event="NodeReady"
Jul 13 21:12:18.253397 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.253288    2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 13 21:12:18.304319 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.304287    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8h742"]
Jul 13 21:12:18.338117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.338088    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-pddgf"]
Jul 13 21:12:18.338263 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.338187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.341146 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.341107    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 13 21:12:18.341298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.341243    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-btz4g\""
Jul 13 21:12:18.341388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.341374    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 13 21:12:18.352641 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.352624    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pddgf"]
Jul 13 21:12:18.352751 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.352664    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8h742"]
Jul 13 21:12:18.352813 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.352749    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.355753 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.355735    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 13 21:12:18.355857 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.355783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 13 21:12:18.355857 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.355798    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 13 21:12:18.356030 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.356018    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gcwv8\""
Jul 13 21:12:18.369370 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.369346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fddf\" (UniqueName: \"kubernetes.io/projected/70ee3ac3-a96e-4c24-b043-653eedc55141-kube-api-access-9fddf\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.369459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.369374    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70ee3ac3-a96e-4c24-b043-653eedc55141-metrics-tls\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.369459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.369405    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jbmr\" (UniqueName: \"kubernetes.io/projected/dce379ae-5c3e-42b8-a6d9-8b2ff403682a-kube-api-access-9jbmr\") pod \"ingress-canary-pddgf\" (UID: \"dce379ae-5c3e-42b8-a6d9-8b2ff403682a\") " pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.369541 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.369462    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce379ae-5c3e-42b8-a6d9-8b2ff403682a-cert\") pod \"ingress-canary-pddgf\" (UID: \"dce379ae-5c3e-42b8-a6d9-8b2ff403682a\") " pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.369541 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.369489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ee3ac3-a96e-4c24-b043-653eedc55141-tmp-dir\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.369541 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.369518    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70ee3ac3-a96e-4c24-b043-653eedc55141-config-volume\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.408281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.408258    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7tw5q"]
Jul 13 21:12:18.428507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.428489    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7tw5q"]
Jul 13 21:12:18.428618 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.428602    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.431091 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.431075    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 13 21:12:18.431196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.431120    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 13 21:12:18.431196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.431120    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 13 21:12:18.431308 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.431120    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hmn24\""
Jul 13 21:12:18.431308 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.431126    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 13 21:12:18.469913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.469891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70ee3ac3-a96e-4c24-b043-653eedc55141-config-volume\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.470021 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.469916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fddf\" (UniqueName: \"kubernetes.io/projected/70ee3ac3-a96e-4c24-b043-653eedc55141-kube-api-access-9fddf\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.470021 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.469937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e425633-a728-4da0-81aa-cf53a4344a2b-crio-socket\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.470021 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.469954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e425633-a728-4da0-81aa-cf53a4344a2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.470021 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.469974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70ee3ac3-a96e-4c24-b043-653eedc55141-metrics-tls\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.470021 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jbmr\" (UniqueName: \"kubernetes.io/projected/dce379ae-5c3e-42b8-a6d9-8b2ff403682a-kube-api-access-9jbmr\") pod \"ingress-canary-pddgf\" (UID: \"dce379ae-5c3e-42b8-a6d9-8b2ff403682a\") " pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.470278 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e425633-a728-4da0-81aa-cf53a4344a2b-data-volume\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.470278 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzxp7\" (UniqueName: \"kubernetes.io/projected/2e425633-a728-4da0-81aa-cf53a4344a2b-kube-api-access-kzxp7\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.470278 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470155    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce379ae-5c3e-42b8-a6d9-8b2ff403682a-cert\") pod \"ingress-canary-pddgf\" (UID: \"dce379ae-5c3e-42b8-a6d9-8b2ff403682a\") " pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.470278 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470204    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ee3ac3-a96e-4c24-b043-653eedc55141-tmp-dir\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.470278 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470268    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e425633-a728-4da0-81aa-cf53a4344a2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.470516 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470468    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/70ee3ac3-a96e-4c24-b043-653eedc55141-config-volume\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.470571 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.470554    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ee3ac3-a96e-4c24-b043-653eedc55141-tmp-dir\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.473806 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.473783    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/70ee3ac3-a96e-4c24-b043-653eedc55141-metrics-tls\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.473888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.473809    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce379ae-5c3e-42b8-a6d9-8b2ff403682a-cert\") pod \"ingress-canary-pddgf\" (UID: \"dce379ae-5c3e-42b8-a6d9-8b2ff403682a\") " pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.477026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.477005    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jbmr\" (UniqueName: \"kubernetes.io/projected/dce379ae-5c3e-42b8-a6d9-8b2ff403682a-kube-api-access-9jbmr\") pod \"ingress-canary-pddgf\" (UID: \"dce379ae-5c3e-42b8-a6d9-8b2ff403682a\") " pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.477654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.477635    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fddf\" (UniqueName: \"kubernetes.io/projected/70ee3ac3-a96e-4c24-b043-653eedc55141-kube-api-access-9fddf\") pod \"dns-default-8h742\" (UID: \"70ee3ac3-a96e-4c24-b043-653eedc55141\") " pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.570747 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.570719    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e425633-a728-4da0-81aa-cf53a4344a2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.570874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.570758    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e425633-a728-4da0-81aa-cf53a4344a2b-crio-socket\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.570874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.570774    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e425633-a728-4da0-81aa-cf53a4344a2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.570874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.570819    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e425633-a728-4da0-81aa-cf53a4344a2b-data-volume\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.570874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.570834    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzxp7\" (UniqueName: \"kubernetes.io/projected/2e425633-a728-4da0-81aa-cf53a4344a2b-kube-api-access-kzxp7\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.571091 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.570941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2e425633-a728-4da0-81aa-cf53a4344a2b-crio-socket\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.571251 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.571233    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2e425633-a728-4da0-81aa-cf53a4344a2b-data-volume\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.571404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.571384    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2e425633-a728-4da0-81aa-cf53a4344a2b-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.572920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.572899    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2e425633-a728-4da0-81aa-cf53a4344a2b-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.578804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.578782    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzxp7\" (UniqueName: \"kubernetes.io/projected/2e425633-a728-4da0-81aa-cf53a4344a2b-kube-api-access-kzxp7\") pod \"insights-runtime-extractor-7tw5q\" (UID: \"2e425633-a728-4da0-81aa-cf53a4344a2b\") " pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.647322 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.647297    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:18.660347 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.660326    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-pddgf"
Jul 13 21:12:18.737300 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.737200    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7tw5q"
Jul 13 21:12:18.864502 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.862764    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-pddgf"]
Jul 13 21:12:18.867706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.867612    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8h742"]
Jul 13 21:12:18.872152 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:18.872129    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ee3ac3_a96e_4c24_b043_653eedc55141.slice/crio-b939c002cab4e5ea2b48c1e50599e7e98ad33c31a01f4f9f5abb08a99cee39a6 WatchSource:0}: Error finding container b939c002cab4e5ea2b48c1e50599e7e98ad33c31a01f4f9f5abb08a99cee39a6: Status 404 returned error can't find the container with id b939c002cab4e5ea2b48c1e50599e7e98ad33c31a01f4f9f5abb08a99cee39a6
Jul 13 21:12:18.909251 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:18.909223    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7tw5q"]
Jul 13 21:12:18.912443 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:18.912414    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e425633_a728_4da0_81aa_cf53a4344a2b.slice/crio-46362f5910262953dd9f203e8c584148a97d0c46e9bd0ef03de8763a3106233b WatchSource:0}: Error finding container 46362f5910262953dd9f203e8c584148a97d0c46e9bd0ef03de8763a3106233b: Status 404 returned error can't find the container with id 46362f5910262953dd9f203e8c584148a97d0c46e9bd0ef03de8763a3106233b
Jul 13 21:12:19.395796 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:19.395761    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tw5q" event={"ID":"2e425633-a728-4da0-81aa-cf53a4344a2b","Type":"ContainerStarted","Data":"46362f5910262953dd9f203e8c584148a97d0c46e9bd0ef03de8763a3106233b"}
Jul 13 21:12:19.396813 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:19.396795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pddgf" event={"ID":"dce379ae-5c3e-42b8-a6d9-8b2ff403682a","Type":"ContainerStarted","Data":"d6899799beb49b71b69646d18917b4ffd299f05f14bf901f137ca3694399cb82"}
Jul 13 21:12:19.397730 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:19.397711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8h742" event={"ID":"70ee3ac3-a96e-4c24-b043-653eedc55141","Type":"ContainerStarted","Data":"b939c002cab4e5ea2b48c1e50599e7e98ad33c31a01f4f9f5abb08a99cee39a6"}
Jul 13 21:12:20.071849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.070845    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:20.071849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.071247    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:20.071849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.071494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:20.077178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.076384    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:12:20.077178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.076639    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vxg2k\""
Jul 13 21:12:20.077178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.076817    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:12:20.077178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.076850    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:12:20.077178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.076993    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t2zb8\""
Jul 13 21:12:20.077178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.077119    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:12:20.259691 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.258691    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76d56c76b6-fdbpn"]
Jul 13 21:12:20.282767 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.281712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d56c76b6-fdbpn"]
Jul 13 21:12:20.282767 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.281825    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.285495    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.285512    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.285818    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.285910    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.285818    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.286020    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.286036    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-m6npp\""
Jul 13 21:12:20.286340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.286104    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 13 21:12:20.387138 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.386983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-serving-cert\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.387138 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.387024    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-service-ca\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.387138 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.387073    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-oauth-serving-cert\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.387138 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.387132    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-config\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.387456 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.387179    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-oauth-config\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.387456 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.387208    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vxwk\" (UniqueName: \"kubernetes.io/projected/58533e47-f4ce-4336-bf6e-b91ab1da9b99-kube-api-access-4vxwk\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.488195    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-oauth-config\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.488243    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vxwk\" (UniqueName: \"kubernetes.io/projected/58533e47-f4ce-4336-bf6e-b91ab1da9b99-kube-api-access-4vxwk\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.488319    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-serving-cert\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.488346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-service-ca\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.488374    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-oauth-serving-cert\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.488425    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-config\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.489855 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.489173    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-service-ca\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.490136 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.490087    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-config\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.490136 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.490124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-oauth-serving-cert\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.493856 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.493771    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-oauth-config\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.494280 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.494258    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-serving-cert\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.503415 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.503375    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vxwk\" (UniqueName: \"kubernetes.io/projected/58533e47-f4ce-4336-bf6e-b91ab1da9b99-kube-api-access-4vxwk\") pod \"console-76d56c76b6-fdbpn\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") " pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.596816 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.596611    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:20.769184 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:20.769148    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76d56c76b6-fdbpn"]
Jul 13 21:12:20.773201 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:20.773126    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58533e47_f4ce_4336_bf6e_b91ab1da9b99.slice/crio-e7594dffdf4744feaa256e52f1b9bd9ac20bf1228c2ff396231064d982aa2a5a WatchSource:0}: Error finding container e7594dffdf4744feaa256e52f1b9bd9ac20bf1228c2ff396231064d982aa2a5a: Status 404 returned error can't find the container with id e7594dffdf4744feaa256e52f1b9bd9ac20bf1228c2ff396231064d982aa2a5a
Jul 13 21:12:21.404674 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:21.404640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d56c76b6-fdbpn" event={"ID":"58533e47-f4ce-4336-bf6e-b91ab1da9b99","Type":"ContainerStarted","Data":"e7594dffdf4744feaa256e52f1b9bd9ac20bf1228c2ff396231064d982aa2a5a"}
Jul 13 21:12:24.827520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:24.827483    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"]
Jul 13 21:12:24.843420 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:24.843393    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"]
Jul 13 21:12:24.843545 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:24.843494    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:24.846111 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:24.846089    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 13 21:12:24.846261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:24.846127    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-kspv2\""
Jul 13 21:12:24.925524 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:24.925489    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9996cccb-036b-4b6e-ab93-7ce6b25d658c-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-sd96x\" (UID: \"9996cccb-036b-4b6e-ab93-7ce6b25d658c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:25.026090 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:25.026034    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9996cccb-036b-4b6e-ab93-7ce6b25d658c-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-sd96x\" (UID: \"9996cccb-036b-4b6e-ab93-7ce6b25d658c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:25.029884 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:25.029858    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/9996cccb-036b-4b6e-ab93-7ce6b25d658c-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-sd96x\" (UID: \"9996cccb-036b-4b6e-ab93-7ce6b25d658c\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:25.152843 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:25.152755    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:25.553085 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:25.552900    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"]
Jul 13 21:12:25.557627 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:25.557606    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9996cccb_036b_4b6e_ab93_7ce6b25d658c.slice/crio-fd49913fc25f70e65ac6eae11c5ade3dbc45f9266022b4adf6558dc5c32acd52 WatchSource:0}: Error finding container fd49913fc25f70e65ac6eae11c5ade3dbc45f9266022b4adf6558dc5c32acd52: Status 404 returned error can't find the container with id fd49913fc25f70e65ac6eae11c5ade3dbc45f9266022b4adf6558dc5c32acd52
Jul 13 21:12:26.418742 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:26.418457    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-pddgf" event={"ID":"dce379ae-5c3e-42b8-a6d9-8b2ff403682a","Type":"ContainerStarted","Data":"49914eba37d331b88c7b14ee6ff9190ec3583af9e4dde92456a3dfb54c166259"}
Jul 13 21:12:26.420112 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:26.420085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8h742" event={"ID":"70ee3ac3-a96e-4c24-b043-653eedc55141","Type":"ContainerStarted","Data":"6bcf9b41c824c73f23a15337894541c64c25ce62c1e46bce66bb6ab32512835c"}
Jul 13 21:12:26.421331 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:26.421310    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x" event={"ID":"9996cccb-036b-4b6e-ab93-7ce6b25d658c","Type":"ContainerStarted","Data":"fd49913fc25f70e65ac6eae11c5ade3dbc45f9266022b4adf6558dc5c32acd52"}
Jul 13 21:12:26.433439 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:26.433359    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-pddgf" podStartSLOduration=1.914804049 podStartE2EDuration="8.433340814s" totalImagesPullingTime="6.518536765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:26.432705073 +0000 UTC m=+53.945454815" watchObservedRunningTime="2026-07-13 21:12:26.433340814 +0000 UTC m=+53.946090535"
Jul 13 21:12:27.428197 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:27.428025    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d56c76b6-fdbpn" event={"ID":"58533e47-f4ce-4336-bf6e-b91ab1da9b99","Type":"ContainerStarted","Data":"6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48"}
Jul 13 21:12:27.429861 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:27.429798    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tw5q" event={"ID":"2e425633-a728-4da0-81aa-cf53a4344a2b","Type":"ContainerStarted","Data":"14da667867f989387e508928d4af86684b21aac9069b6560456e976d3a3c0f10"}
Jul 13 21:12:27.432557 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:27.432452    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8h742" event={"ID":"70ee3ac3-a96e-4c24-b043-653eedc55141","Type":"ContainerStarted","Data":"b123d0afb1caf1fd85f3648fb6d770acdb9f964393158c82a23026062817cd2e"}
Jul 13 21:12:27.432663 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:27.432576    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:27.445941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:27.445895    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76d56c76b6-fdbpn" podStartSLOduration=1.248320836 podStartE2EDuration="7.445876939s" totalImagesPullingTime="6.197556103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:27.444580741 +0000 UTC m=+54.957330481" watchObservedRunningTime="2026-07-13 21:12:27.445876939 +0000 UTC m=+54.958626683"
Jul 13 21:12:27.463359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:27.463301    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8h742" podStartSLOduration=2.943967956 podStartE2EDuration="9.463281538s" totalImagesPullingTime="6.519313582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:27.462950588 +0000 UTC m=+54.975700330" watchObservedRunningTime="2026-07-13 21:12:27.463281538 +0000 UTC m=+54.976031280"
Jul 13 21:12:28.437762 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.437724    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tw5q" event={"ID":"2e425633-a728-4da0-81aa-cf53a4344a2b","Type":"ContainerStarted","Data":"c6111dffd2629be70db86cb29091d57ea90ade2313a2183524ff5463f78fdcd5"}
Jul 13 21:12:28.439108 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.439085    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x" event={"ID":"9996cccb-036b-4b6e-ab93-7ce6b25d658c","Type":"ContainerStarted","Data":"b1b7aa87dc57dfafde514ccc113e929c90f63a3ef1a636ddde62bc0c0651df63"}
Jul 13 21:12:28.455845 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.455805    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x" podStartSLOduration=2.111910453 podStartE2EDuration="4.455790305s" totalImagesPullingTime="2.343879852s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:28.455104999 +0000 UTC m=+55.967854741" watchObservedRunningTime="2026-07-13 21:12:28.455790305 +0000 UTC m=+55.968540046"
Jul 13 21:12:28.845346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.845311    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-659fd4cf5f-qjwvs"]
Jul 13 21:12:28.847692 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.847670    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.858594 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.858569    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 13 21:12:28.861801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.861780    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659fd4cf5f-qjwvs"]
Jul 13 21:12:28.955202 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-service-ca\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.955391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955209    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fpn2\" (UniqueName: \"kubernetes.io/projected/c2178be2-771b-44e6-b3c5-a0a0ae707c94-kube-api-access-6fpn2\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.955391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955242    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-config\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.955391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955340    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-oauth-config\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.955391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-oauth-serving-cert\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.955391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955382    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-trusted-ca-bundle\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:28.955659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:28.955413    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-serving-cert\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.056846 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.056802    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-service-ca\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.056853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fpn2\" (UniqueName: \"kubernetes.io/projected/c2178be2-771b-44e6-b3c5-a0a0ae707c94-kube-api-access-6fpn2\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.056882    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-config\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.056938    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-oauth-config\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.056960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-oauth-serving-cert\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.056985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-trusted-ca-bundle\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.057024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-serving-cert\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057741 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.057687    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-service-ca\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057868 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.057804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-oauth-serving-cert\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.057974 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.057931    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-config\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.058698 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.058634    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-trusted-ca-bundle\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.062082 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.062043    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-oauth-config\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.062240 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.062220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-serving-cert\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.066147 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.066122    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fpn2\" (UniqueName: \"kubernetes.io/projected/c2178be2-771b-44e6-b3c5-a0a0ae707c94-kube-api-access-6fpn2\") pod \"console-659fd4cf5f-qjwvs\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") " pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.162819 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.162737    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:29.303523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.303487    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-659fd4cf5f-qjwvs"]
Jul 13 21:12:29.306838 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:29.306806    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2178be2_771b_44e6_b3c5_a0a0ae707c94.slice/crio-d18eb252e799467b0037ae27fe9c5bd9a8ea1c9bf1e2734a5c03f689bcd8a969 WatchSource:0}: Error finding container d18eb252e799467b0037ae27fe9c5bd9a8ea1c9bf1e2734a5c03f689bcd8a969: Status 404 returned error can't find the container with id d18eb252e799467b0037ae27fe9c5bd9a8ea1c9bf1e2734a5c03f689bcd8a969
Jul 13 21:12:29.445137 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.445028    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659fd4cf5f-qjwvs" event={"ID":"c2178be2-771b-44e6-b3c5-a0a0ae707c94","Type":"ContainerStarted","Data":"31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450"}
Jul 13 21:12:29.445137 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.445101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659fd4cf5f-qjwvs" event={"ID":"c2178be2-771b-44e6-b3c5-a0a0ae707c94","Type":"ContainerStarted","Data":"d18eb252e799467b0037ae27fe9c5bd9a8ea1c9bf1e2734a5c03f689bcd8a969"}
Jul 13 21:12:29.445623 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.445296    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:29.451196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.451169    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-sd96x"
Jul 13 21:12:29.462756 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.462714    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-659fd4cf5f-qjwvs" podStartSLOduration=1.462696739 podStartE2EDuration="1.462696739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:29.462389604 +0000 UTC m=+56.975139347" watchObservedRunningTime="2026-07-13 21:12:29.462696739 +0000 UTC m=+56.975446478"
Jul 13 21:12:29.885657 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.885605    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-l8rbn"]
Jul 13 21:12:29.890582 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.890556    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:29.893129 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.893104    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Jul 13 21:12:29.893269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.893104    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 13 21:12:29.893269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.893232    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Jul 13 21:12:29.893269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.893248    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 13 21:12:29.894004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.893982    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-h29z4\""
Jul 13 21:12:29.894117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.894002    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 13 21:12:29.896368 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.896325    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-l8rbn"]
Jul 13 21:12:29.963798 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.963766    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:29.963971 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.963855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zspjt\" (UniqueName: \"kubernetes.io/projected/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-kube-api-access-zspjt\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:29.964026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.963964    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:29.964026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:29.963996    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.064797 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.064759    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zspjt\" (UniqueName: \"kubernetes.io/projected/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-kube-api-access-zspjt\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.064953 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.064930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.065003 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.064968    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.065037 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.065014    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.065792 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.065742    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-metrics-client-ca\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.068112 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.068067    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.068112 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.068076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.072760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.072732    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zspjt\" (UniqueName: \"kubernetes.io/projected/44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1-kube-api-access-zspjt\") pod \"prometheus-operator-64785b6675-l8rbn\" (UID: \"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1\") " pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.201866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.201782    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn"
Jul 13 21:12:30.598481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.598358    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:30.598481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.598413    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:12:30.599522 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.599500    2564 patch_prober.go:28] interesting pod/console-76d56c76b6-fdbpn container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
Jul 13 21:12:30.599621 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.599545    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76d56c76b6-fdbpn" podUID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" containerName="console" probeResult="failure" output="Get \"https://10.134.0.9:8443/health\": dial tcp 10.134.0.9:8443: connect: connection refused"
Jul 13 21:12:30.627089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:30.627042    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-l8rbn"]
Jul 13 21:12:30.631078 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:30.631042    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44ac09b0_b3cb_47b0_823a_b6c5bfeebcf1.slice/crio-414da0efc6719485198c600195b4eca9d05cfa34ffd6e030e2422e4661c4c6ee WatchSource:0}: Error finding container 414da0efc6719485198c600195b4eca9d05cfa34ffd6e030e2422e4661c4c6ee: Status 404 returned error can't find the container with id 414da0efc6719485198c600195b4eca9d05cfa34ffd6e030e2422e4661c4c6ee
Jul 13 21:12:31.453283 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:31.453244    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn" event={"ID":"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1","Type":"ContainerStarted","Data":"414da0efc6719485198c600195b4eca9d05cfa34ffd6e030e2422e4661c4c6ee"}
Jul 13 21:12:31.455476 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:31.455446    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7tw5q" event={"ID":"2e425633-a728-4da0-81aa-cf53a4344a2b","Type":"ContainerStarted","Data":"2cf339b9fa51326275e05a6144fbab5a427b991c422796e58dd7648bd48f8371"}
Jul 13 21:12:31.474192 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:31.474147    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7tw5q" podStartSLOduration=1.989215335 podStartE2EDuration="13.474134074s" totalImagesPullingTime="11.484918739s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:12:18 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:31.472802533 +0000 UTC m=+58.985552280" watchObservedRunningTime="2026-07-13 21:12:31.474134074 +0000 UTC m=+58.986883815"
Jul 13 21:12:32.460845 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:32.460806    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn" event={"ID":"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1","Type":"ContainerStarted","Data":"a7d6f5b83bceb224b8e864de4774f241446ef811ad9e1da4b3ef50f2ff031442"}
Jul 13 21:12:32.461229 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:32.460870    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn" event={"ID":"44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1","Type":"ContainerStarted","Data":"cc82ecad2a63d01ec56c3aac88f466d2599f69fd13f78cafecd8b90365519f2a"}
Jul 13 21:12:32.477390 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:32.477331    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-l8rbn" podStartSLOduration=2.13082943 podStartE2EDuration="3.477313325s" totalImagesPullingTime="1.346483895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:32.476686963 +0000 UTC m=+59.989436729" watchObservedRunningTime="2026-07-13 21:12:32.477313325 +0000 UTC m=+59.990063068"
Jul 13 21:12:33.357049 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:33.357021    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6dxmp"
Jul 13 21:12:34.239986 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.239955    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"]
Jul 13 21:12:34.266475 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.266441    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"]
Jul 13 21:12:34.266623 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.266562    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.269568 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.269546    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Jul 13 21:12:34.269568 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.269554    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-p9f2h\""
Jul 13 21:12:34.269755 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.269628    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Jul 13 21:12:34.287634 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.287598    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8vmk6"]
Jul 13 21:12:34.297851 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.297824    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48ac65a8-654e-4ec2-9d05-836e9aa41865-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.297992 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.297874    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.297992 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.297906    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.297992 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.297938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqhff\" (UniqueName: \"kubernetes.io/projected/48ac65a8-654e-4ec2-9d05-836e9aa41865-kube-api-access-nqhff\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.305858 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.305833    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"]
Jul 13 21:12:34.305971 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.305952    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.308963 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.308927    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 13 21:12:34.309222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.309203    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-nphg5\""
Jul 13 21:12:34.309331 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.309259    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 13 21:12:34.310078 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.309594    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 13 21:12:34.327012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.326989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"]
Jul 13 21:12:34.327130 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.327094    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.329730 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.329712    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 13 21:12:34.329828 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.329729    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 13 21:12:34.329828 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.329789    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 13 21:12:34.329939 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.329832    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-pp7hx\""
Jul 13 21:12:34.398505 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398468    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-sys\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.398505 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398511    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cnwr\" (UniqueName: \"kubernetes.io/projected/abccc841-bd4a-4d6c-927b-014259705317-kube-api-access-5cnwr\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.398706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-root\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.398706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.398706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398568    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-tls\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.398706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398627    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.398706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398691    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqhff\" (UniqueName: \"kubernetes.io/projected/48ac65a8-654e-4ec2-9d05-836e9aa41865-kube-api-access-nqhff\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.398853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78c01e4b-3698-4b79-99e1-65474d804a3d-metrics-client-ca\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.398853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398745    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.398853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398771    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.398853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398837    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-accelerators-collector-config\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.398980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/abccc841-bd4a-4d6c-927b-014259705317-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.398980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398892    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48ac65a8-654e-4ec2-9d05-836e9aa41865-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.398980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abccc841-bd4a-4d6c-927b-014259705317-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.398980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398937    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-wtmp\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.398980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.398975    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-textfile\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.399255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.399005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.399255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.399077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.399255 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:34.399124    2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
Jul 13 21:12:34.399255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.399141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzlzx\" (UniqueName: \"kubernetes.io/projected/78c01e4b-3698-4b79-99e1-65474d804a3d-kube-api-access-hzlzx\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.399255 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:34.399203    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-tls podName:48ac65a8-654e-4ec2-9d05-836e9aa41865 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:34.899178493 +0000 UTC m=+62.411928213 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-hdplc" (UID: "48ac65a8-654e-4ec2-9d05-836e9aa41865") : secret "openshift-state-metrics-tls" not found
Jul 13 21:12:34.399625 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.399607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48ac65a8-654e-4ec2-9d05-836e9aa41865-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.401100 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.401072    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.406690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.406669    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqhff\" (UniqueName: \"kubernetes.io/projected/48ac65a8-654e-4ec2-9d05-836e9aa41865-kube-api-access-nqhff\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.500002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.499913    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-textfile\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.499960    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.499985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzlzx\" (UniqueName: \"kubernetes.io/projected/78c01e4b-3698-4b79-99e1-65474d804a3d-kube-api-access-hzlzx\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500005    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-sys\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500024    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cnwr\" (UniqueName: \"kubernetes.io/projected/abccc841-bd4a-4d6c-927b-014259705317-kube-api-access-5cnwr\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500039    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-root\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500077    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500095    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-tls\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500118    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-sys\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500164    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-root\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:34.500212    2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500216    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78c01e4b-3698-4b79-99e1-65474d804a3d-metrics-client-ca\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500269    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:34.500278    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-tls podName:78c01e4b-3698-4b79-99e1-65474d804a3d nodeName:}" failed. No retries permitted until 2026-07-13 21:12:35.000260928 +0000 UTC m=+62.513010677 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-tls") pod "node-exporter-8vmk6" (UID: "78c01e4b-3698-4b79-99e1-65474d804a3d") : secret "node-exporter-tls" not found
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-textfile\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500376    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-accelerators-collector-config\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500408    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/abccc841-bd4a-4d6c-927b-014259705317-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500439    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abccc841-bd4a-4d6c-927b-014259705317-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500479    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-wtmp\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500642    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-wtmp\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500762    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78c01e4b-3698-4b79-99e1-65474d804a3d-metrics-client-ca\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.500972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.500844    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/abccc841-bd4a-4d6c-927b-014259705317-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.501346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.501115    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.501346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.501276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-accelerators-collector-config\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.501346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.501311    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/abccc841-bd4a-4d6c-927b-014259705317-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.502635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.502609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.502728 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.502697    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.502774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.502733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/abccc841-bd4a-4d6c-927b-014259705317-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.508259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.508220    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzlzx\" (UniqueName: \"kubernetes.io/projected/78c01e4b-3698-4b79-99e1-65474d804a3d-kube-api-access-hzlzx\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:34.508443 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.508426    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cnwr\" (UniqueName: \"kubernetes.io/projected/abccc841-bd4a-4d6c-927b-014259705317-kube-api-access-5cnwr\") pod \"kube-state-metrics-785778ff74-2n2mj\" (UID: \"abccc841-bd4a-4d6c-927b-014259705317\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.636112 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.636050    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"
Jul 13 21:12:34.789188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.789095    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-2n2mj"]
Jul 13 21:12:34.791785 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:34.791763    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabccc841_bd4a_4d6c_927b_014259705317.slice/crio-ec87f7ab909615cd29153e8fd5651dc8fe8a6c58f7e3fd4d795d0836a469a74a WatchSource:0}: Error finding container ec87f7ab909615cd29153e8fd5651dc8fe8a6c58f7e3fd4d795d0836a469a74a: Status 404 returned error can't find the container with id ec87f7ab909615cd29153e8fd5651dc8fe8a6c58f7e3fd4d795d0836a469a74a
Jul 13 21:12:34.903958 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.903922    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:34.906322 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:34.906291    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/48ac65a8-654e-4ec2-9d05-836e9aa41865-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-hdplc\" (UID: \"48ac65a8-654e-4ec2-9d05-836e9aa41865\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:35.005006 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.004966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-tls\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:35.007271 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.007247    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/78c01e4b-3698-4b79-99e1-65474d804a3d-node-exporter-tls\") pod \"node-exporter-8vmk6\" (UID: \"78c01e4b-3698-4b79-99e1-65474d804a3d\") " pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:35.175073 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.175020    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"
Jul 13 21:12:35.218736 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.218708    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8vmk6"
Jul 13 21:12:35.249497 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:35.249467    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78c01e4b_3698_4b79_99e1_65474d804a3d.slice/crio-36dc9a33c141f3b87df68bfb46fd4fc117c612b72c4a43add6728c779b92cae4 WatchSource:0}: Error finding container 36dc9a33c141f3b87df68bfb46fd4fc117c612b72c4a43add6728c779b92cae4: Status 404 returned error can't find the container with id 36dc9a33c141f3b87df68bfb46fd4fc117c612b72c4a43add6728c779b92cae4
Jul 13 21:12:35.327348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.327318    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:12:35.332756 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.332739    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.335072 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335040    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 13 21:12:35.335318 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335303    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 13 21:12:35.335425 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335314    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 13 21:12:35.335425 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335381    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 13 21:12:35.335425 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335314    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 13 21:12:35.335598 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335452    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 13 21:12:35.335598 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335539    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 13 21:12:35.335752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335646    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 13 21:12:35.335752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335654    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 13 21:12:35.335752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.335705    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-c484b\""
Jul 13 21:12:35.340654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.339922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc"]
Jul 13 21:12:35.343102 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:35.343074    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ac65a8_654e_4ec2_9d05_836e9aa41865.slice/crio-0d1f5eb696da949c80c63120f5ad69397fb32bc271219e189e3c2a2a592c5180 WatchSource:0}: Error finding container 0d1f5eb696da949c80c63120f5ad69397fb32bc271219e189e3c2a2a592c5180: Status 404 returned error can't find the container with id 0d1f5eb696da949c80c63120f5ad69397fb32bc271219e189e3c2a2a592c5180
Jul 13 21:12:35.347063 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.347022    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:12:35.408666 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408634    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.408835 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408681    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpmdl\" (UniqueName: \"kubernetes.io/projected/955d53bc-3343-4682-bdbc-bee9a9637ebd-kube-api-access-xpmdl\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.408835 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408723    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.408835 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408767    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408831    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408916    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408942    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.408976    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955d53bc-3343-4682-bdbc-bee9a9637ebd-config-out\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.409001    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-config-volume\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.409093    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955d53bc-3343-4682-bdbc-bee9a9637ebd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.409137    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-web-config\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.409163    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.409267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.409245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.476008 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.475953    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vmk6" event={"ID":"78c01e4b-3698-4b79-99e1-65474d804a3d","Type":"ContainerStarted","Data":"36dc9a33c141f3b87df68bfb46fd4fc117c612b72c4a43add6728c779b92cae4"}
Jul 13 21:12:35.478250 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.478199    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc" event={"ID":"48ac65a8-654e-4ec2-9d05-836e9aa41865","Type":"ContainerStarted","Data":"7ecaa605a20853d43393b4109f617a185490d390645d50aa47f159ec875c9c15"}
Jul 13 21:12:35.478425 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.478275    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc" event={"ID":"48ac65a8-654e-4ec2-9d05-836e9aa41865","Type":"ContainerStarted","Data":"0d1f5eb696da949c80c63120f5ad69397fb32bc271219e189e3c2a2a592c5180"}
Jul 13 21:12:35.479797 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.479739    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj" event={"ID":"abccc841-bd4a-4d6c-927b-014259705317","Type":"ContainerStarted","Data":"ec87f7ab909615cd29153e8fd5651dc8fe8a6c58f7e3fd4d795d0836a469a74a"}
Jul 13 21:12:35.510387 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510356    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955d53bc-3343-4682-bdbc-bee9a9637ebd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510533 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510398    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-web-config\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510533 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510416    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510646 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510616    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510700 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510666    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510747 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpmdl\" (UniqueName: \"kubernetes.io/projected/955d53bc-3343-4682-bdbc-bee9a9637ebd-kube-api-access-xpmdl\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510799 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510799 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510777    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510894 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510803    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510894 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510840    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.510894 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:35.510869    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-trusted-ca-bundle podName:955d53bc-3343-4682-bdbc-bee9a9637ebd nodeName:}" failed. No retries permitted until 2026-07-13 21:12:36.01084407 +0000 UTC m=+63.523593811 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "955d53bc-3343-4682-bdbc-bee9a9637ebd") : configmap references non-existent config key: ca-bundle.crt
Jul 13 21:12:35.511076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510912    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.511076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510947    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955d53bc-3343-4682-bdbc-bee9a9637ebd-config-out\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.511076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.510977    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-config-volume\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.511076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.511032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.512348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.512035    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.515974 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.515657    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.518771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.517120    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-web-config\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.518771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.517415    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.518771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.517817    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/955d53bc-3343-4682-bdbc-bee9a9637ebd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.518771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.518267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.518771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.518552    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/955d53bc-3343-4682-bdbc-bee9a9637ebd-config-out\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.518771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.518729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.519154 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.518979    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.519154 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.519027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/955d53bc-3343-4682-bdbc-bee9a9637ebd-config-volume\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:35.524190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:35.524168    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpmdl\" (UniqueName: \"kubernetes.io/projected/955d53bc-3343-4682-bdbc-bee9a9637ebd-kube-api-access-xpmdl\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:36.017303 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.017263    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:36.018267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.018232    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/955d53bc-3343-4682-bdbc-bee9a9637ebd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"955d53bc-3343-4682-bdbc-bee9a9637ebd\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:36.245277 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.245240    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 13 21:12:36.489891 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.489780    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc" event={"ID":"48ac65a8-654e-4ec2-9d05-836e9aa41865","Type":"ContainerStarted","Data":"11487f211a57b7abc0f1980b9a7d3c56c1cf3ea22b2d5b47dcae3e90289fa354"}
Jul 13 21:12:36.493084 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.492964    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj" event={"ID":"abccc841-bd4a-4d6c-927b-014259705317","Type":"ContainerStarted","Data":"7a74ef652152169579790dd5b26236bfbde7c1bde4c5da5abe0f92b048e6605f"}
Jul 13 21:12:36.496224 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.495784    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vmk6" event={"ID":"78c01e4b-3698-4b79-99e1-65474d804a3d","Type":"ContainerStarted","Data":"ddc52fe4eabc9a53b41778c4fceb5f506a2260c9bf80f5180d68c6697a670816"}
Jul 13 21:12:36.525247 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:36.525139    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 13 21:12:36.769790 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:36.769754    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod955d53bc_3343_4682_bdbc_bee9a9637ebd.slice/crio-5cdf09fb56b8b7717adf83e60ee8eff267b28c783f6e81b2803c3d55c29198cb WatchSource:0}: Error finding container 5cdf09fb56b8b7717adf83e60ee8eff267b28c783f6e81b2803c3d55c29198cb: Status 404 returned error can't find the container with id 5cdf09fb56b8b7717adf83e60ee8eff267b28c783f6e81b2803c3d55c29198cb
Jul 13 21:12:37.229774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.229701    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"]
Jul 13 21:12:37.232452 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.232433    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.234951 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.234928    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Jul 13 21:12:37.235102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.235084    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-6ngmlauq18a9k\""
Jul 13 21:12:37.235174 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.235116    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Jul 13 21:12:37.235232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.235114    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Jul 13 21:12:37.235232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.235202    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Jul 13 21:12:37.235232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.235212    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-htmgn\""
Jul 13 21:12:37.235447 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.235434    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Jul 13 21:12:37.246723 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.246698    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"]
Jul 13 21:12:37.329207 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329175    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-grpc-tls\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329342 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329236    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5kf2\" (UniqueName: \"kubernetes.io/projected/5d723943-824c-4935-8516-409e5470786e-kube-api-access-v5kf2\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329342 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329308    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329439 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329343    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-tls\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329439 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329364    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329439 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329404    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329439 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329436    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.329580 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.329451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d723943-824c-4935-8516-409e5470786e-metrics-client-ca\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.430735 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.430699    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.430877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.430754    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.430877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.430783    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d723943-824c-4935-8516-409e5470786e-metrics-client-ca\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.430877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.430829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-grpc-tls\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.431115 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.431090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5kf2\" (UniqueName: \"kubernetes.io/projected/5d723943-824c-4935-8516-409e5470786e-kube-api-access-v5kf2\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.431300 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.431273    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.431422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.431313    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-tls\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.431422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.431338    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.431820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.431792    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d723943-824c-4935-8516-409e5470786e-metrics-client-ca\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.433807 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.433784    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-grpc-tls\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.434255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.434210    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.434777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.434646    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.434777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.434736    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.435404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.435380    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-tls\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.435404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.435396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5d723943-824c-4935-8516-409e5470786e-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.438524 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.438503    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5kf2\" (UniqueName: \"kubernetes.io/projected/5d723943-824c-4935-8516-409e5470786e-kube-api-access-v5kf2\") pod \"thanos-querier-7f5b6944bb-bb5lh\" (UID: \"5d723943-824c-4935-8516-409e5470786e\") " pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.442328 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.442309    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8h742"
Jul 13 21:12:37.504635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.504538    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc" event={"ID":"48ac65a8-654e-4ec2-9d05-836e9aa41865","Type":"ContainerStarted","Data":"edf82eefa6e99a5b174e3bedad4b1f33cd906172bc3c91fa419c1cf349c4fd4c"}
Jul 13 21:12:37.507226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.507196    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj" event={"ID":"abccc841-bd4a-4d6c-927b-014259705317","Type":"ContainerStarted","Data":"113d84f85fcea83886643fb4c9bcf28c5516f94073bc9760f26239e0421d58bb"}
Jul 13 21:12:37.507393 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.507250    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj" event={"ID":"abccc841-bd4a-4d6c-927b-014259705317","Type":"ContainerStarted","Data":"5c333987d8c29d79bcabb478d2b71fed322006c346aae454296e6457f3643dbb"}
Jul 13 21:12:37.509239 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.509101    2564 generic.go:356] "Generic (PLEG): container finished" podID="78c01e4b-3698-4b79-99e1-65474d804a3d" containerID="ddc52fe4eabc9a53b41778c4fceb5f506a2260c9bf80f5180d68c6697a670816" exitCode=0
Jul 13 21:12:37.509239 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.509185    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vmk6" event={"ID":"78c01e4b-3698-4b79-99e1-65474d804a3d","Type":"ContainerDied","Data":"ddc52fe4eabc9a53b41778c4fceb5f506a2260c9bf80f5180d68c6697a670816"}
Jul 13 21:12:37.510648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.510574    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"5cdf09fb56b8b7717adf83e60ee8eff267b28c783f6e81b2803c3d55c29198cb"}
Jul 13 21:12:37.523207 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.523152    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-hdplc" podStartSLOduration=2.220461729 podStartE2EDuration="3.523132665s" totalImagesPullingTime="1.302670936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:37.520919927 +0000 UTC m=+65.033669676" watchObservedRunningTime="2026-07-13 21:12:37.523132665 +0000 UTC m=+65.035882406"
Jul 13 21:12:37.540899 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.540862    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-2n2mj" podStartSLOduration=1.994033663 podStartE2EDuration="3.540846307s" totalImagesPullingTime="1.546812644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:37.540179899 +0000 UTC m=+65.052929654" watchObservedRunningTime="2026-07-13 21:12:37.540846307 +0000 UTC m=+65.053596048"
Jul 13 21:12:37.545797 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.545773    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:37.694635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.694098    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"]
Jul 13 21:12:37.698532 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:37.698506    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d723943_824c_4935_8516_409e5470786e.slice/crio-96ba235af9961bc1c732286d78b4cc43999daff7804a477f2913b6182754f711 WatchSource:0}: Error finding container 96ba235af9961bc1c732286d78b4cc43999daff7804a477f2913b6182754f711: Status 404 returned error can't find the container with id 96ba235af9961bc1c732286d78b4cc43999daff7804a477f2913b6182754f711
Jul 13 21:12:37.734432 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.734409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:37.734538 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.734473    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:37.736803 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.736783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 13 21:12:37.736883 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.736783    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 13 21:12:37.747173 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.747155    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f76ffc5f-2c98-42fb-996f-a62e56716196-metrics-certs\") pod \"network-metrics-daemon-q7js4\" (UID: \"f76ffc5f-2c98-42fb-996f-a62e56716196\") " pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:37.747328 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.747305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f911bedb-e01d-4f73-a947-89e2fd905115-original-pull-secret\") pod \"global-pull-secret-syncer-czhjv\" (UID: \"f911bedb-e01d-4f73-a947-89e2fd905115\") " pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:37.792093 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.792016    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vxg2k\""
Jul 13 21:12:37.799855 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.799827    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7js4"
Jul 13 21:12:37.801896 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.801874    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-czhjv"
Jul 13 21:12:37.935404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.935369    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:37.938127 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.938102    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 13 21:12:37.948161 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.948134    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 13 21:12:37.959366 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:37.959326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4mwr\" (UniqueName: \"kubernetes.io/projected/bd10968a-4ce3-40e8-9e06-26b1bca018dc-kube-api-access-m4mwr\") pod \"network-check-target-qs78m\" (UID: \"bd10968a-4ce3-40e8-9e06-26b1bca018dc\") " pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:38.101630 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.101607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7js4"]
Jul 13 21:12:38.112314 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.112294    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-t2zb8\""
Jul 13 21:12:38.114048 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.114031    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-czhjv"]
Jul 13 21:12:38.120178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.120160    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:38.134990 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:38.134959    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf76ffc5f_2c98_42fb_996f_a62e56716196.slice/crio-163ffb7a7e345f51cef5978d1f8a5dd7194502e7aa159e63fdcb7c072ba8c0d7 WatchSource:0}: Error finding container 163ffb7a7e345f51cef5978d1f8a5dd7194502e7aa159e63fdcb7c072ba8c0d7: Status 404 returned error can't find the container with id 163ffb7a7e345f51cef5978d1f8a5dd7194502e7aa159e63fdcb7c072ba8c0d7
Jul 13 21:12:38.135207 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:38.135189    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf911bedb_e01d_4f73_a947_89e2fd905115.slice/crio-55c671cd8ce0b48cadc2fa831ef2e04625ee796fe051845a0d3e12b2cd615895 WatchSource:0}: Error finding container 55c671cd8ce0b48cadc2fa831ef2e04625ee796fe051845a0d3e12b2cd615895: Status 404 returned error can't find the container with id 55c671cd8ce0b48cadc2fa831ef2e04625ee796fe051845a0d3e12b2cd615895
Jul 13 21:12:38.240270 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.240242    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qs78m"]
Jul 13 21:12:38.243577 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:38.243551    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd10968a_4ce3_40e8_9e06_26b1bca018dc.slice/crio-478de14ca3297e44feaca2e608cbe1812d2ab33cf54788363fbb077716280e22 WatchSource:0}: Error finding container 478de14ca3297e44feaca2e608cbe1812d2ab33cf54788363fbb077716280e22: Status 404 returned error can't find the container with id 478de14ca3297e44feaca2e608cbe1812d2ab33cf54788363fbb077716280e22
Jul 13 21:12:38.516695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.516657    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7js4" event={"ID":"f76ffc5f-2c98-42fb-996f-a62e56716196","Type":"ContainerStarted","Data":"163ffb7a7e345f51cef5978d1f8a5dd7194502e7aa159e63fdcb7c072ba8c0d7"}
Jul 13 21:12:38.518136 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.518106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-czhjv" event={"ID":"f911bedb-e01d-4f73-a947-89e2fd905115","Type":"ContainerStarted","Data":"55c671cd8ce0b48cadc2fa831ef2e04625ee796fe051845a0d3e12b2cd615895"}
Jul 13 21:12:38.519492 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.519458    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"96ba235af9961bc1c732286d78b4cc43999daff7804a477f2913b6182754f711"}
Jul 13 21:12:38.521948 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.521926    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vmk6" event={"ID":"78c01e4b-3698-4b79-99e1-65474d804a3d","Type":"ContainerStarted","Data":"a263db3c7bd775b821d1cfcb9edb3e6d72e5f41331e0f9bc2422659e19f411f3"}
Jul 13 21:12:38.521948 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.521967    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8vmk6" event={"ID":"78c01e4b-3698-4b79-99e1-65474d804a3d","Type":"ContainerStarted","Data":"91347d4a20e9bf79b811388b3b3df9a0a5c20260c118356607eb9ff7ee5de92f"}
Jul 13 21:12:38.523652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.523624    2564 generic.go:356] "Generic (PLEG): container finished" podID="955d53bc-3343-4682-bdbc-bee9a9637ebd" containerID="6de8610cfa37a91d06985cb9996b2d023a8361f5a045a496b3272264c46a27d6" exitCode=0
Jul 13 21:12:38.523744 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.523705    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerDied","Data":"6de8610cfa37a91d06985cb9996b2d023a8361f5a045a496b3272264c46a27d6"}
Jul 13 21:12:38.525033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.525013    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qs78m" event={"ID":"bd10968a-4ce3-40e8-9e06-26b1bca018dc","Type":"ContainerStarted","Data":"478de14ca3297e44feaca2e608cbe1812d2ab33cf54788363fbb077716280e22"}
Jul 13 21:12:38.547642 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.547616    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5c9d58844f-z7jz7"]
Jul 13 21:12:38.549964 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.549913    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8vmk6" podStartSLOduration=3.365350309 podStartE2EDuration="4.549895972s" totalImagesPullingTime="1.091103963s" totalInitContainerRuntime="93.4417ms" isStatefulPod=true podCreationTimestamp="2026-07-13 21:12:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:38.548535422 +0000 UTC m=+66.061285164" watchObservedRunningTime="2026-07-13 21:12:38.549895972 +0000 UTC m=+66.062645716"
Jul 13 21:12:38.551075 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.551029    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.553847 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.553826    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 13 21:12:38.553971 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.553947    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-5h76q55t7ipaq\""
Jul 13 21:12:38.554093 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.554035    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-cvnp8\""
Jul 13 21:12:38.554170 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.554097    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Jul 13 21:12:38.554223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.554184    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Jul 13 21:12:38.554223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.554208    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Jul 13 21:12:38.563201 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.563153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c9d58844f-z7jz7"]
Jul 13 21:12:38.646319 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.645756    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-secret-metrics-server-client-certs\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.646319 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.645802    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-secret-metrics-server-tls\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.646319 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.645838    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-client-ca-bundle\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.646319 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.646165    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c97d73-447f-4146-b2b8-554f40a536ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.646649 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.646571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b2c97d73-447f-4146-b2b8-554f40a536ad-metrics-server-audit-profiles\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.646649 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.646636    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjpvb\" (UniqueName: \"kubernetes.io/projected/b2c97d73-447f-4146-b2b8-554f40a536ad-kube-api-access-zjpvb\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.646751 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.646697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b2c97d73-447f-4146-b2b8-554f40a536ad-audit-log\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.747910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.747799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-secret-metrics-server-client-certs\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.747910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.747841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-secret-metrics-server-tls\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.747910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.747872    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-client-ca-bundle\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.748216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.747917    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c97d73-447f-4146-b2b8-554f40a536ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.748216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.747974    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b2c97d73-447f-4146-b2b8-554f40a536ad-metrics-server-audit-profiles\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.748216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.748008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjpvb\" (UniqueName: \"kubernetes.io/projected/b2c97d73-447f-4146-b2b8-554f40a536ad-kube-api-access-zjpvb\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.748216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.748035    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b2c97d73-447f-4146-b2b8-554f40a536ad-audit-log\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.748671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.748443    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b2c97d73-447f-4146-b2b8-554f40a536ad-audit-log\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.750254 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.749480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b2c97d73-447f-4146-b2b8-554f40a536ad-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.750254 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.750208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b2c97d73-447f-4146-b2b8-554f40a536ad-metrics-server-audit-profiles\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.754571 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.754522    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-client-ca-bundle\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.755227 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.755202    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-secret-metrics-server-client-certs\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.758287 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.758262    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b2c97d73-447f-4146-b2b8-554f40a536ad-secret-metrics-server-tls\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.758933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.758770    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjpvb\" (UniqueName: \"kubernetes.io/projected/b2c97d73-447f-4146-b2b8-554f40a536ad-kube-api-access-zjpvb\") pod \"metrics-server-5c9d58844f-z7jz7\" (UID: \"b2c97d73-447f-4146-b2b8-554f40a536ad\") " pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:38.865485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:38.865410    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:39.027012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.025492    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"]
Jul 13 21:12:39.028576 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.028549    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.031360 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.031316    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-w22nx\""
Jul 13 21:12:39.031555 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.031525    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 13 21:12:39.035703 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.035658    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"]
Jul 13 21:12:39.152996 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.152796    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fbns\" (UniqueName: \"kubernetes.io/projected/3f481cc0-58bb-4f3f-a762-123940420847-kube-api-access-8fbns\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.152996 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.152846    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f481cc0-58bb-4f3f-a762-123940420847-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.163646 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.163616    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:39.163786 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.163697    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:39.165639 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.165404    2564 patch_prober.go:28] interesting pod/console-659fd4cf5f-qjwvs container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused" start-of-body=
Jul 13 21:12:39.165639 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.165508    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-659fd4cf5f-qjwvs" podUID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" containerName="console" probeResult="failure" output="Get \"https://10.134.0.11:8443/health\": dial tcp 10.134.0.11:8443: connect: connection refused"
Jul 13 21:12:39.254376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.254340    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fbns\" (UniqueName: \"kubernetes.io/projected/3f481cc0-58bb-4f3f-a762-123940420847-kube-api-access-8fbns\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.254521 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.254409    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f481cc0-58bb-4f3f-a762-123940420847-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.254929 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:39.254909    2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 13 21:12:39.255019 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:12:39.254987    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f481cc0-58bb-4f3f-a762-123940420847-monitoring-plugin-cert podName:3f481cc0-58bb-4f3f-a762-123940420847 nodeName:}" failed. No retries permitted until 2026-07-13 21:12:39.754965579 +0000 UTC m=+67.267715312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/3f481cc0-58bb-4f3f-a762-123940420847-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-dkvks" (UID: "3f481cc0-58bb-4f3f-a762-123940420847") : secret "monitoring-plugin-cert" not found
Jul 13 21:12:39.264823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.264774    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fbns\" (UniqueName: \"kubernetes.io/projected/3f481cc0-58bb-4f3f-a762-123940420847-kube-api-access-8fbns\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.348718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.348686    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5c9d58844f-z7jz7"]
Jul 13 21:12:39.759452 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.759392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f481cc0-58bb-4f3f-a762-123940420847-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.762647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.762621    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/3f481cc0-58bb-4f3f-a762-123940420847-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-dkvks\" (UID: \"3f481cc0-58bb-4f3f-a762-123940420847\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:39.943253 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:39.943216    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:40.070722 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:40.070686    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2c97d73_447f_4146_b2b8_554f40a536ad.slice/crio-21c102dfd6cb0bbad870ab96ce12c630d953cf272e5d8c0822ea76b6483093be WatchSource:0}: Error finding container 21c102dfd6cb0bbad870ab96ce12c630d953cf272e5d8c0822ea76b6483093be: Status 404 returned error can't find the container with id 21c102dfd6cb0bbad870ab96ce12c630d953cf272e5d8c0822ea76b6483093be
Jul 13 21:12:40.264162 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.263792    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76d56c76b6-fdbpn"]
Jul 13 21:12:40.270545 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.270507    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"]
Jul 13 21:12:40.291197 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.291148    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d8f47fcc7-jzr7h"]
Jul 13 21:12:40.294532 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.294462    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.311657 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.309047    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8f47fcc7-jzr7h"]
Jul 13 21:12:40.366367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-trusted-ca-bundle\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.366367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366326    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-serving-cert\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.366367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366359    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-config\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.366694 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-service-ca\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.366694 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-oauth-config\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.366694 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366562    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgfkh\" (UniqueName: \"kubernetes.io/projected/ec22cfb9-e3ec-4503-a236-347c2058fe9c-kube-api-access-wgfkh\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.366694 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.366589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-oauth-serving-cert\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.467925    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-trusted-ca-bundle\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.467990    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-serving-cert\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.468090    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-config\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.468130    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-service-ca\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.468173    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-oauth-config\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.468218    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgfkh\" (UniqueName: \"kubernetes.io/projected/ec22cfb9-e3ec-4503-a236-347c2058fe9c-kube-api-access-wgfkh\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.468244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-oauth-serving-cert\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.468990 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.468960    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-oauth-serving-cert\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.469570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.469542    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-config\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.470470 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.470427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-service-ca\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.470780 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.470694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-trusted-ca-bundle\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.472634 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.472609    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-oauth-config\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.477660 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.477581    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-serving-cert\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.480783 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.480729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgfkh\" (UniqueName: \"kubernetes.io/projected/ec22cfb9-e3ec-4503-a236-347c2058fe9c-kube-api-access-wgfkh\") pod \"console-5d8f47fcc7-jzr7h\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") " pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.495942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.493588    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 21:12:40.508344 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.508309    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.512933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.512906    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-tpzrz\""
Jul 13 21:12:40.513101 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.512950    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 13 21:12:40.513101 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.512966    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-17bj0nlvtl5oi\""
Jul 13 21:12:40.513220 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.512920    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 13 21:12:40.513679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.513651    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 13 21:12:40.513918 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.513777    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 13 21:12:40.513918 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.513857    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 13 21:12:40.514087 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.513969    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 13 21:12:40.515854 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.515229    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 13 21:12:40.515854 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.515383    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 13 21:12:40.515854 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.515551    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 13 21:12:40.515854 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.515645    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 13 21:12:40.515854 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.515703    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 13 21:12:40.515854 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.515802    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 13 21:12:40.516559 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.516537    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 13 21:12:40.517157 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.517138    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 13 21:12:40.526521 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.526442    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 21:12:40.544274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.544031    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7js4" event={"ID":"f76ffc5f-2c98-42fb-996f-a62e56716196","Type":"ContainerStarted","Data":"d6689c0663c74079b4301d9168ad430a98cda82fd02f9bfc1826027cbcb81af4"}
Jul 13 21:12:40.547239 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.547204    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"9d057bad0c60c9221a2057389633d49ba323f210f892201a21b00131ea002810"}
Jul 13 21:12:40.549996 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.549955    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7" event={"ID":"b2c97d73-447f-4146-b2b8-554f40a536ad","Type":"ContainerStarted","Data":"21c102dfd6cb0bbad870ab96ce12c630d953cf272e5d8c0822ea76b6483093be"}
Jul 13 21:12:40.569668 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569596    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569668 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569648    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569668 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569708    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569738    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e66bf19-1726-42a7-a7db-15141c2f2db3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569790    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569818    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-config\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569852    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569877    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569902    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e66bf19-1726-42a7-a7db-15141c2f2db3-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.569967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569954    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.569979    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.570030    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.570080    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hq6cb\" (UniqueName: \"kubernetes.io/projected/3e66bf19-1726-42a7-a7db-15141c2f2db3-kube-api-access-hq6cb\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.570205    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.570297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.570385    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.571033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.570422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.612373 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.612326    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:40.671814 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.671733    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.671827    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hq6cb\" (UniqueName: \"kubernetes.io/projected/3e66bf19-1726-42a7-a7db-15141c2f2db3-kube-api-access-hq6cb\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.671884    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.671916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.671949    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.671984    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672029    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672081    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e66bf19-1726-42a7-a7db-15141c2f2db3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672418    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672485    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-config\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672520    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672544    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672562    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672571    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e66bf19-1726-42a7-a7db-15141c2f2db3-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672629    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672657    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.672717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672694    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.673497 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672756    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.673497 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.672910    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.673928 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.673682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.673928 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.673805    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.679605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.678208    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-config\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.679605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.678480    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.679809 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.679663    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/3e66bf19-1726-42a7-a7db-15141c2f2db3-config-out\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.681481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.680419    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.681481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.681081    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.681641 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.681574    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/3e66bf19-1726-42a7-a7db-15141c2f2db3-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.682776 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.682753    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/3e66bf19-1726-42a7-a7db-15141c2f2db3-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.683526 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.683478    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.684730 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.684703    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.684850 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.684827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.685529 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.685506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.685755 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.685722    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-web-config\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.686446 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.686406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hq6cb\" (UniqueName: \"kubernetes.io/projected/3e66bf19-1726-42a7-a7db-15141c2f2db3-kube-api-access-hq6cb\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.687654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.687629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/3e66bf19-1726-42a7-a7db-15141c2f2db3-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"3e66bf19-1726-42a7-a7db-15141c2f2db3\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.825258 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:40.824943    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:40.856992 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:40.856958    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f481cc0_58bb_4f3f_a762_123940420847.slice/crio-2de7485e5f337049c69b93d336123ace4858c23cb9afa99ef7a453e5d4208516 WatchSource:0}: Error finding container 2de7485e5f337049c69b93d336123ace4858c23cb9afa99ef7a453e5d4208516: Status 404 returned error can't find the container with id 2de7485e5f337049c69b93d336123ace4858c23cb9afa99ef7a453e5d4208516
Jul 13 21:12:41.055485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.055435    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d8f47fcc7-jzr7h"]
Jul 13 21:12:41.061791 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:41.061749    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec22cfb9_e3ec_4503_a236_347c2058fe9c.slice/crio-5f4491eebbde959052df0e5b0911b4623c2174fe22b8f12148bb06405fd70138 WatchSource:0}: Error finding container 5f4491eebbde959052df0e5b0911b4623c2174fe22b8f12148bb06405fd70138: Status 404 returned error can't find the container with id 5f4491eebbde959052df0e5b0911b4623c2174fe22b8f12148bb06405fd70138
Jul 13 21:12:41.091205 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.090187    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 13 21:12:41.285353 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:12:41.285313    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e66bf19_1726_42a7_a7db_15141c2f2db3.slice/crio-37b30ee10c7f7a7afdee18b46cba81dc31028ca0ca53b34ae9ed7c1c5439c0f2 WatchSource:0}: Error finding container 37b30ee10c7f7a7afdee18b46cba81dc31028ca0ca53b34ae9ed7c1c5439c0f2: Status 404 returned error can't find the container with id 37b30ee10c7f7a7afdee18b46cba81dc31028ca0ca53b34ae9ed7c1c5439c0f2
Jul 13 21:12:41.557028 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.556882    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks" event={"ID":"3f481cc0-58bb-4f3f-a762-123940420847","Type":"ContainerStarted","Data":"2de7485e5f337049c69b93d336123ace4858c23cb9afa99ef7a453e5d4208516"}
Jul 13 21:12:41.560017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.559944    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7js4" event={"ID":"f76ffc5f-2c98-42fb-996f-a62e56716196","Type":"ContainerStarted","Data":"ec3a56a35f72e91541fec5c65fdc60f31359a6d54c72a009d90e9241c4338cd5"}
Jul 13 21:12:41.561967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.561936    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"37b30ee10c7f7a7afdee18b46cba81dc31028ca0ca53b34ae9ed7c1c5439c0f2"}
Jul 13 21:12:41.565182 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.565151    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"11647e6affd0f5fba987ad1ddeb1a4ebbaefc02dfd00e54843388fbbacbcff1c"}
Jul 13 21:12:41.566936 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:41.566856    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8f47fcc7-jzr7h" event={"ID":"ec22cfb9-e3ec-4503-a236-347c2058fe9c","Type":"ContainerStarted","Data":"5f4491eebbde959052df0e5b0911b4623c2174fe22b8f12148bb06405fd70138"}
Jul 13 21:12:45.592121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.592089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8f47fcc7-jzr7h" event={"ID":"ec22cfb9-e3ec-4503-a236-347c2058fe9c","Type":"ContainerStarted","Data":"b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566"}
Jul 13 21:12:45.593460 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.593440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks" event={"ID":"3f481cc0-58bb-4f3f-a762-123940420847","Type":"ContainerStarted","Data":"bb8a35698297fdecdd5d448a09cace247eb58fb6a1bfaa7021f2ebf328b6250a"}
Jul 13 21:12:45.593620 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.593606    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:45.594936 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.594914    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7" event={"ID":"b2c97d73-447f-4146-b2b8-554f40a536ad","Type":"ContainerStarted","Data":"1f0b2c3c60136cadd93c6c16f1ea5e341733352ac4f80a14a6197907676132a1"}
Jul 13 21:12:45.596797 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.596779    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qs78m" event={"ID":"bd10968a-4ce3-40e8-9e06-26b1bca018dc","Type":"ContainerStarted","Data":"25a919a726f460f401b765e9fb66cbf56c6ee59043651407b919847fee96974d"}
Jul 13 21:12:45.596902 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.596867    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:12:45.601105 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.600118    2564 generic.go:356] "Generic (PLEG): container finished" podID="3e66bf19-1726-42a7-a7db-15141c2f2db3" containerID="3b9f70efe72b8735d66f3a66ae08dacfc6bb556b850ea0b0224b428ea163aa96" exitCode=0
Jul 13 21:12:45.601105 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.600213    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerDied","Data":"3b9f70efe72b8735d66f3a66ae08dacfc6bb556b850ea0b0224b428ea163aa96"}
Jul 13 21:12:45.603181 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.603158    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks"
Jul 13 21:12:45.605379 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.605357    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-czhjv" event={"ID":"f911bedb-e01d-4f73-a947-89e2fd905115","Type":"ContainerStarted","Data":"b754d31c7e74f1eb53a8fc93c1f8c47edfca81355ea947f2d7c8e6e5035651e1"}
Jul 13 21:12:45.607760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.607741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"098962a81fca22c9ab029bb43313cdc67a15ef650fea28f9c9d20c8a80b0dc4e"}
Jul 13 21:12:45.607833 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.607787    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"22a046f39c3b48f54d5297200d92bce930ea336bb09033b3edcfac13f210b40c"}
Jul 13 21:12:45.609746 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.609698    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d8f47fcc7-jzr7h" podStartSLOduration=5.609686387 podStartE2EDuration="5.609686387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:45.608482998 +0000 UTC m=+73.121232740" watchObservedRunningTime="2026-07-13 21:12:45.609686387 +0000 UTC m=+73.122436128"
Jul 13 21:12:45.610017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.609983    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q7js4" podStartSLOduration=70.661555273 podStartE2EDuration="1m12.6099758s" totalImagesPullingTime="1.948420527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:41.577220168 +0000 UTC m=+69.089969910" watchObservedRunningTime="2026-07-13 21:12:45.6099758 +0000 UTC m=+73.122725542"
Jul 13 21:12:45.611796 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.611777    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"31f62f59d85e8268f638c776555c064b73576f220033f99efd0ff1d57046a05d"}
Jul 13 21:12:45.611873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.611824    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"5c9c9adc6f6131a436613854dd31fc22602de1524c076cef28ecc5a1d9159214"}
Jul 13 21:12:45.611873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.611848    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"3bbf6cbec135db720c31201df0af80293a452f2e2ea4eedc71dbc8c00b2ffd23"}
Jul 13 21:12:45.611873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.611872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"52edc600c66b4e371a7a405c6c787d40f3941b9aa2504a1e689bcd1e174119d0"}
Jul 13 21:12:45.611965 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.611894    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"43ef5392457c83c56ccdc19271dd83f9305eb0dbaae7264c58b7853116b3c28a"}
Jul 13 21:12:45.611965 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.611918    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"955d53bc-3343-4682-bdbc-bee9a9637ebd","Type":"ContainerStarted","Data":"3837e2a58e728d48686aab607ce0e7aa4cac719f3961693adc4d7d3ce776a1d4"}
Jul 13 21:12:45.657700 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.657655    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7" podStartSLOduration=3.175400561 podStartE2EDuration="7.657638168s" totalImagesPullingTime="4.482237607s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:45.65683893 +0000 UTC m=+73.169588673" watchObservedRunningTime="2026-07-13 21:12:45.657638168 +0000 UTC m=+73.170387974"
Jul 13 21:12:45.672443 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.672286    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-dkvks" podStartSLOduration=2.967295129 podStartE2EDuration="6.672270058s" totalImagesPullingTime="3.704974929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:45.671103586 +0000 UTC m=+73.183853328" watchObservedRunningTime="2026-07-13 21:12:45.672270058 +0000 UTC m=+73.185019807"
Jul 13 21:12:45.686957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.686912    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qs78m" podStartSLOduration=66.381465342 podStartE2EDuration="1m12.686896079s" totalImagesPullingTime="6.305430737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:45.685173118 +0000 UTC m=+73.197922861" watchObservedRunningTime="2026-07-13 21:12:45.686896079 +0000 UTC m=+73.199645820"
Jul 13 21:12:45.706410 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.706357    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-czhjv" podStartSLOduration=66.291870914 podStartE2EDuration="1m12.70633516s" totalImagesPullingTime="6.414464246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:11:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:45.705631343 +0000 UTC m=+73.218381084" watchObservedRunningTime="2026-07-13 21:12:45.70633516 +0000 UTC m=+73.219084903"
Jul 13 21:12:45.730442 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:45.730356    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.282619805 podStartE2EDuration="10.730340728s" totalImagesPullingTime="7.428847869s" totalInitContainerRuntime="18.873054ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:45.728527654 +0000 UTC m=+73.241277395" watchObservedRunningTime="2026-07-13 21:12:45.730340728 +0000 UTC m=+73.243090469"
Jul 13 21:12:46.622518 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:46.622469    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"733408703bc7f521829415bfad85e58fcc4b006615e47f5117cf1d909f2da86f"}
Jul 13 21:12:46.622995 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:46.622548    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" event={"ID":"5d723943-824c-4935-8516-409e5470786e","Type":"ContainerStarted","Data":"20e4b22b217f94691a7afb79aa9fb59d4de914cf30012d0cfc268203d68116ef"}
Jul 13 21:12:46.645346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:46.645287    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh" podStartSLOduration=6.408000389 podStartE2EDuration="9.645271107s" totalImagesPullingTime="3.237270718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:37 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:46.64391928 +0000 UTC m=+74.156669021" watchObservedRunningTime="2026-07-13 21:12:46.645271107 +0000 UTC m=+74.158020848"
Jul 13 21:12:47.631716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:47.630937    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:47.636534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:47.636515    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7f5b6944bb-bb5lh"
Jul 13 21:12:49.168004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:49.167972    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:49.172126 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:49.172104    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:12:49.642571 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:49.642539    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"c50854e0e3f6f171ea38d5025220d25d93fb27a10ad751f3b12716c03cb9109a"}
Jul 13 21:12:49.642675 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:49.642612    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"9edb43d34febece682481a2f608da4d4fc8da9bff49d4ebb76539f6f766ecf85"}
Jul 13 21:12:49.642675 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:49.642650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"f63356c0e164c7618af91b27dfe1e39308b9e752edb8483897e44a37c085082d"}
Jul 13 21:12:49.642803 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:49.642686    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"c87af0f5eec984463b257bcd199c6285a660277486a4468d11a80eec1e191b8a"}
Jul 13 21:12:50.613148 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.613106    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:50.613528 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.613203    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:50.618036 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.618012    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:50.650397 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.650362    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"b60fd1cc334fe540730b8fa9260eeadc8d14c5620618c51a1480ae73b02a94a5"}
Jul 13 21:12:50.650557 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.650440    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"3e66bf19-1726-42a7-a7db-15141c2f2db3","Type":"ContainerStarted","Data":"9b852690ea8a04c4074f89fddb9d8078a851ac1b89792c05f62da312e07566a1"}
Jul 13 21:12:50.654384 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.654361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:12:50.678132 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.678030    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=6.9322743110000005 podStartE2EDuration="10.678008677s" totalImagesPullingTime="3.711738095s" totalInitContainerRuntime="33.996271ms" isStatefulPod=false podCreationTimestamp="2026-07-13 21:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:12:50.676010399 +0000 UTC m=+78.188760183" watchObservedRunningTime="2026-07-13 21:12:50.678008677 +0000 UTC m=+78.190758418"
Jul 13 21:12:50.723178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.723146    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659fd4cf5f-qjwvs"]
Jul 13 21:12:50.825627 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:50.825596    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:12:58.866629 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:58.866489    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:12:58.867312 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:12:58.866959    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:13:05.299903 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.299855    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76d56c76b6-fdbpn" podUID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" containerName="console" containerID="cri-o://6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48" gracePeriod=15
Jul 13 21:13:05.540232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.540209    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:13:05.614377 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614349    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-serving-cert\") pod \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") "
Jul 13 21:13:05.614377 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614380    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-service-ca\") pod \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") "
Jul 13 21:13:05.614599 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614412    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-config\") pod \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") "
Jul 13 21:13:05.614599 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614445    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-oauth-serving-cert\") pod \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") "
Jul 13 21:13:05.614599 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614485    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vxwk\" (UniqueName: \"kubernetes.io/projected/58533e47-f4ce-4336-bf6e-b91ab1da9b99-kube-api-access-4vxwk\") pod \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") "
Jul 13 21:13:05.614599 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614507    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-oauth-config\") pod \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\" (UID: \"58533e47-f4ce-4336-bf6e-b91ab1da9b99\") "
Jul 13 21:13:05.614839 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614816    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-service-ca" (OuterVolumeSpecName: "service-ca") pod "58533e47-f4ce-4336-bf6e-b91ab1da9b99" (UID: "58533e47-f4ce-4336-bf6e-b91ab1da9b99"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:05.614892 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614839    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-config" (OuterVolumeSpecName: "console-config") pod "58533e47-f4ce-4336-bf6e-b91ab1da9b99" (UID: "58533e47-f4ce-4336-bf6e-b91ab1da9b99"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:05.614928 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.614890    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "58533e47-f4ce-4336-bf6e-b91ab1da9b99" (UID: "58533e47-f4ce-4336-bf6e-b91ab1da9b99"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:05.616851 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.616819    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "58533e47-f4ce-4336-bf6e-b91ab1da9b99" (UID: "58533e47-f4ce-4336-bf6e-b91ab1da9b99"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:13:05.616950 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.616862    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58533e47-f4ce-4336-bf6e-b91ab1da9b99-kube-api-access-4vxwk" (OuterVolumeSpecName: "kube-api-access-4vxwk") pod "58533e47-f4ce-4336-bf6e-b91ab1da9b99" (UID: "58533e47-f4ce-4336-bf6e-b91ab1da9b99"). InnerVolumeSpecName "kube-api-access-4vxwk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:13:05.616950 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.616872    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "58533e47-f4ce-4336-bf6e-b91ab1da9b99" (UID: "58533e47-f4ce-4336-bf6e-b91ab1da9b99"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:13:05.715507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.715478    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:05.715507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.715508    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:05.715654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.715523    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:05.715654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.715537    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/58533e47-f4ce-4336-bf6e-b91ab1da9b99-oauth-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:05.715654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.715549    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vxwk\" (UniqueName: \"kubernetes.io/projected/58533e47-f4ce-4336-bf6e-b91ab1da9b99-kube-api-access-4vxwk\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:05.715654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.715563    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/58533e47-f4ce-4336-bf6e-b91ab1da9b99-console-oauth-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:05.717538 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.717507    2564 generic.go:356] "Generic (PLEG): container finished" podID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" containerID="6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48" exitCode=0
Jul 13 21:13:05.717651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.717554    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76d56c76b6-fdbpn"
Jul 13 21:13:05.717651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.717596    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d56c76b6-fdbpn" event={"ID":"58533e47-f4ce-4336-bf6e-b91ab1da9b99","Type":"ContainerDied","Data":"6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48"}
Jul 13 21:13:05.717651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.717640    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76d56c76b6-fdbpn" event={"ID":"58533e47-f4ce-4336-bf6e-b91ab1da9b99","Type":"ContainerDied","Data":"e7594dffdf4744feaa256e52f1b9bd9ac20bf1228c2ff396231064d982aa2a5a"}
Jul 13 21:13:05.717755 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.717660    2564 scope.go:117] "RemoveContainer" containerID="6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48"
Jul 13 21:13:05.726699 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.726687    2564 scope.go:117] "RemoveContainer" containerID="6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48"
Jul 13 21:13:05.726943 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:13:05.726922    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48\": container with ID starting with 6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48 not found: ID does not exist" containerID="6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48"
Jul 13 21:13:05.727011 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.726955    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48"} err="failed to get container status \"6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48\": rpc error: code = NotFound desc = could not find container \"6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48\": container with ID starting with 6a7610fc3389197ea2bd3ec89f9c9794768397e618b6e8312871b4e061fb7e48 not found: ID does not exist"
Jul 13 21:13:05.738121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.738100    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76d56c76b6-fdbpn"]
Jul 13 21:13:05.741987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:05.741967    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76d56c76b6-fdbpn"]
Jul 13 21:13:07.075424 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:07.075394    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" path="/var/lib/kubelet/pods/58533e47-f4ce-4336-bf6e-b91ab1da9b99/volumes"
Jul 13 21:13:12.196156 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:12.196129    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pddgf_dce379ae-5c3e-42b8-a6d9-8b2ff403682a/serve-healthcheck-canary/0.log"
Jul 13 21:13:15.753301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:15.753258    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-659fd4cf5f-qjwvs" podUID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" containerName="console" containerID="cri-o://31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450" gracePeriod=15
Jul 13 21:13:16.024455 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.024386    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:13:16.103513 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103489    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-oauth-config\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.103671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103601    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-service-ca\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.103671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103633    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-serving-cert\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.103671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103670    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-config\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.103840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103712    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fpn2\" (UniqueName: \"kubernetes.io/projected/c2178be2-771b-44e6-b3c5-a0a0ae707c94-kube-api-access-6fpn2\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.103840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103727    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-oauth-serving-cert\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.103840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.103780    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-trusted-ca-bundle\") pod \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\" (UID: \"c2178be2-771b-44e6-b3c5-a0a0ae707c94\") "
Jul 13 21:13:16.104122 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.104006    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-service-ca" (OuterVolumeSpecName: "service-ca") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:16.104207 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.104121    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-config" (OuterVolumeSpecName: "console-config") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:16.104281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.104252    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:16.104364 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.104341    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:13:16.105728 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.105698    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:13:16.105832 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.105760    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:13:16.106121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.106083    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2178be2-771b-44e6-b3c5-a0a0ae707c94-kube-api-access-6fpn2" (OuterVolumeSpecName: "kube-api-access-6fpn2") pod "c2178be2-771b-44e6-b3c5-a0a0ae707c94" (UID: "c2178be2-771b-44e6-b3c5-a0a0ae707c94"). InnerVolumeSpecName "kube-api-access-6fpn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:13:16.205162 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205134    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.205162 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205156    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fpn2\" (UniqueName: \"kubernetes.io/projected/c2178be2-771b-44e6-b3c5-a0a0ae707c94-kube-api-access-6fpn2\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.205301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205166    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-oauth-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.205301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205176    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-trusted-ca-bundle\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.205301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205184    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-oauth-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.205301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205192    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c2178be2-771b-44e6-b3c5-a0a0ae707c94-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.205301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.205200    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2178be2-771b-44e6-b3c5-a0a0ae707c94-console-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:13:16.625696 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.625664    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qs78m"
Jul 13 21:13:16.763753 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.763716    2564 generic.go:356] "Generic (PLEG): container finished" podID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" containerID="31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450" exitCode=0
Jul 13 21:13:16.764229 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.763767    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-659fd4cf5f-qjwvs"
Jul 13 21:13:16.764229 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.763769    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659fd4cf5f-qjwvs" event={"ID":"c2178be2-771b-44e6-b3c5-a0a0ae707c94","Type":"ContainerDied","Data":"31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450"}
Jul 13 21:13:16.764229 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.763895    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-659fd4cf5f-qjwvs" event={"ID":"c2178be2-771b-44e6-b3c5-a0a0ae707c94","Type":"ContainerDied","Data":"d18eb252e799467b0037ae27fe9c5bd9a8ea1c9bf1e2734a5c03f689bcd8a969"}
Jul 13 21:13:16.764229 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.763922    2564 scope.go:117] "RemoveContainer" containerID="31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450"
Jul 13 21:13:16.772798 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.772784    2564 scope.go:117] "RemoveContainer" containerID="31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450"
Jul 13 21:13:16.773047 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:13:16.773027    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450\": container with ID starting with 31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450 not found: ID does not exist" containerID="31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450"
Jul 13 21:13:16.773153 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.773074    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450"} err="failed to get container status \"31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450\": rpc error: code = NotFound desc = could not find container \"31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450\": container with ID starting with 31ca6d7b824e1e67608aa52e3b1e5dd64a53606f4709bcb9f660f30c3f03b450 not found: ID does not exist"
Jul 13 21:13:16.784064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.784028    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-659fd4cf5f-qjwvs"]
Jul 13 21:13:16.785685 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:16.785664    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-659fd4cf5f-qjwvs"]
Jul 13 21:13:17.075215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:17.075185    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" path="/var/lib/kubelet/pods/c2178be2-771b-44e6-b3c5-a0a0ae707c94/volumes"
Jul 13 21:13:18.871404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:18.871374    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:13:18.875134 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:18.875109    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5c9d58844f-z7jz7"
Jul 13 21:13:40.826385 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:40.826346    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:13:40.845633 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:40.845607    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:13:40.890913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:40.890885    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 13 21:13:58.540358 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540320    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"]
Jul 13 21:13:58.540900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540691    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" containerName="console"
Jul 13 21:13:58.540900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540705    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" containerName="console"
Jul 13 21:13:58.540900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540727    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" containerName="console"
Jul 13 21:13:58.540900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540733    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" containerName="console"
Jul 13 21:13:58.540900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540778    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2178be2-771b-44e6-b3c5-a0a0ae707c94" containerName="console"
Jul 13 21:13:58.540900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.540787    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="58533e47-f4ce-4336-bf6e-b91ab1da9b99" containerName="console"
Jul 13 21:13:58.542931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.542916    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.545500 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.545443    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 13 21:13:58.545645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.545513    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 13 21:13:58.545645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.545514    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 13 21:13:58.545645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.545565    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 13 21:13:58.545645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.545514    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-546m7\""
Jul 13 21:13:58.545858 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.545698    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 13 21:13:58.553367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.553342    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 13 21:13:58.555273 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.555248    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"]
Jul 13 21:13:58.657897 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.657864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-metrics-client-ca\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.657904    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-telemeter-client-tls\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.657924    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-federate-client-tls\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.657983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.658070    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.658107    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjgnj\" (UniqueName: \"kubernetes.io/projected/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-kube-api-access-sjgnj\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.658135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-serving-certs-ca-bundle\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.658215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.658174    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-secret-telemeter-client\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.758660 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758631    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.758807 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758683    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.758807 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758712    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sjgnj\" (UniqueName: \"kubernetes.io/projected/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-kube-api-access-sjgnj\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.758807 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-serving-certs-ca-bundle\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.758807 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758779    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-secret-telemeter-client\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.759007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758810    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-metrics-client-ca\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.759007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758848    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-telemeter-client-tls\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.759007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.758874    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-federate-client-tls\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.759709 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.759679    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-metrics-client-ca\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.759836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.759794    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-telemeter-trusted-ca-bundle\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.760227 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.760199    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-serving-certs-ca-bundle\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.761612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.761588    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-federate-client-tls\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.761942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.761919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.762035 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.761920    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-telemeter-client-tls\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.762035 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.761972    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-secret-telemeter-client\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.766412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.766394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjgnj\" (UniqueName: \"kubernetes.io/projected/3cf7bb72-46a0-4ccf-a7d8-60a0d3311591-kube-api-access-sjgnj\") pod \"telemeter-client-cb77b4fcf-hvm97\" (UID: \"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591\") " pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.854952 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.854930    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"
Jul 13 21:13:58.985618 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:58.985582    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-cb77b4fcf-hvm97"]
Jul 13 21:13:58.988639 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:13:58.988616    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf7bb72_46a0_4ccf_a7d8_60a0d3311591.slice/crio-46080bb54ed27de5fd6d3bb70adf829c112b40de0214f4ec60532844b590a620 WatchSource:0}: Error finding container 46080bb54ed27de5fd6d3bb70adf829c112b40de0214f4ec60532844b590a620: Status 404 returned error can't find the container with id 46080bb54ed27de5fd6d3bb70adf829c112b40de0214f4ec60532844b590a620
Jul 13 21:13:59.948698 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:13:59.948664    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97" event={"ID":"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591","Type":"ContainerStarted","Data":"46080bb54ed27de5fd6d3bb70adf829c112b40de0214f4ec60532844b590a620"}
Jul 13 21:14:01.958605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:01.958571    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97" event={"ID":"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591","Type":"ContainerStarted","Data":"2e6ba157ab7bf366081ab9a375490bd7dc1eb6d3f787b37f75ce41ed69b4c31c"}
Jul 13 21:14:01.959020 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:01.958630    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97" event={"ID":"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591","Type":"ContainerStarted","Data":"0d28cc87ed09264cca07aff92af733c1b6285e9cdf8a84eab9902226c6158b96"}
Jul 13 21:14:01.959020 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:01.958649    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97" event={"ID":"3cf7bb72-46a0-4ccf-a7d8-60a0d3311591","Type":"ContainerStarted","Data":"974e57a56338ced2c0650f6c8122fda19675887c12a3a934e006d13022c5eaa1"}
Jul 13 21:14:01.981176 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:01.981123    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-cb77b4fcf-hvm97" podStartSLOduration=1.693640112 podStartE2EDuration="3.981102623s" totalImagesPullingTime="2.287462511s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:13:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:14:01.979441381 +0000 UTC m=+149.492191124" watchObservedRunningTime="2026-07-13 21:14:01.981102623 +0000 UTC m=+149.493852362"
Jul 13 21:14:02.881985 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.881942    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6b64b47bfd-z2j2v"]
Jul 13 21:14:02.884460 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.884429    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.895233 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.895207    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b64b47bfd-z2j2v"]
Jul 13 21:14:02.995009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.994971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-trusted-ca-bundle\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.995520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.995120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-oauth-config\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.995520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.995189    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-service-ca\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.995520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.995235    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-serving-cert\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.995520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.995258    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-console-config\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.995520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.995317    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-oauth-serving-cert\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:02.995520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:02.995469    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvp6t\" (UniqueName: \"kubernetes.io/projected/04bfc789-58d1-4cba-bf3a-023b91389581-kube-api-access-hvp6t\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096625 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096588    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-oauth-config\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096636    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-service-ca\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-serving-cert\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-console-config\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096708    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-oauth-serving-cert\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096747    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvp6t\" (UniqueName: \"kubernetes.io/projected/04bfc789-58d1-4cba-bf3a-023b91389581-kube-api-access-hvp6t\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.096818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.096777    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-trusted-ca-bundle\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.097462 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.097436    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-service-ca\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.097575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.097547    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-oauth-serving-cert\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.097636 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.097546    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-console-config\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.097833 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.097814    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-trusted-ca-bundle\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.099152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.099123    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-serving-cert\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.099249 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.099154    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-oauth-config\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.104074 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.104037    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvp6t\" (UniqueName: \"kubernetes.io/projected/04bfc789-58d1-4cba-bf3a-023b91389581-kube-api-access-hvp6t\") pod \"console-6b64b47bfd-z2j2v\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") " pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.195222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.195132    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:03.320568 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.320539    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6b64b47bfd-z2j2v"]
Jul 13 21:14:03.322411 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:14:03.322389    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04bfc789_58d1_4cba_bf3a_023b91389581.slice/crio-985e4f4cef28a246ba334e1683103ea3d986d17b577e619a885359f2208d9a5a WatchSource:0}: Error finding container 985e4f4cef28a246ba334e1683103ea3d986d17b577e619a885359f2208d9a5a: Status 404 returned error can't find the container with id 985e4f4cef28a246ba334e1683103ea3d986d17b577e619a885359f2208d9a5a
Jul 13 21:14:03.967588 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.967544    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64b47bfd-z2j2v" event={"ID":"04bfc789-58d1-4cba-bf3a-023b91389581","Type":"ContainerStarted","Data":"1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44"}
Jul 13 21:14:03.967825 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.967641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64b47bfd-z2j2v" event={"ID":"04bfc789-58d1-4cba-bf3a-023b91389581","Type":"ContainerStarted","Data":"985e4f4cef28a246ba334e1683103ea3d986d17b577e619a885359f2208d9a5a"}
Jul 13 21:14:03.985133 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:03.985086    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6b64b47bfd-z2j2v" podStartSLOduration=1.985072803 podStartE2EDuration="1.985072803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:14:03.983954499 +0000 UTC m=+151.496704241" watchObservedRunningTime="2026-07-13 21:14:03.985072803 +0000 UTC m=+151.497822533"
Jul 13 21:14:12.766028 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.765994    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b64b47bfd-z2j2v"]
Jul 13 21:14:12.801138 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.801110    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55cc55db57-xjnx5"]
Jul 13 21:14:12.803343 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.803323    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.817525 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.817502    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55cc55db57-xjnx5"]
Jul 13 21:14:12.882933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.882910    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-oauth-serving-cert\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.883032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.882945    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgmwj\" (UniqueName: \"kubernetes.io/projected/cb7f0b85-9a02-4872-b384-0410bd7f9195-kube-api-access-mgmwj\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.883032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.882965    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-trusted-ca-bundle\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.883032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.882982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-serving-cert\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.883203 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.883048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-config\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.883203 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.883135    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-oauth-config\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.883203 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.883182    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-service-ca\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984206 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984177    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-serving-cert\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984213    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-config\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984240    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-oauth-config\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-service-ca\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984295    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-oauth-serving-cert\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984348 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984321    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgmwj\" (UniqueName: \"kubernetes.io/projected/cb7f0b85-9a02-4872-b384-0410bd7f9195-kube-api-access-mgmwj\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.984610 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.984448    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-trusted-ca-bundle\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.985089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.985047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-service-ca\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.985195 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.985047    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-oauth-serving-cert\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.985195 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.985181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-trusted-ca-bundle\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.985364 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.985345    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-config\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.986727 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.986689    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-serving-cert\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.986829 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.986808    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-oauth-config\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:12.993179 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:12.993159    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgmwj\" (UniqueName: \"kubernetes.io/projected/cb7f0b85-9a02-4872-b384-0410bd7f9195-kube-api-access-mgmwj\") pod \"console-55cc55db57-xjnx5\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") " pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:13.112043 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:13.112016    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:13.195281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:13.195248    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:13.236650 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:13.236628    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55cc55db57-xjnx5"]
Jul 13 21:14:13.238735 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:14:13.238706    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb7f0b85_9a02_4872_b384_0410bd7f9195.slice/crio-04f3b9e6096d017bf28edec2e19b342427d5511fe43f0b478bd5992a5fd48c55 WatchSource:0}: Error finding container 04f3b9e6096d017bf28edec2e19b342427d5511fe43f0b478bd5992a5fd48c55: Status 404 returned error can't find the container with id 04f3b9e6096d017bf28edec2e19b342427d5511fe43f0b478bd5992a5fd48c55
Jul 13 21:14:14.010543 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:14.010504    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55cc55db57-xjnx5" event={"ID":"cb7f0b85-9a02-4872-b384-0410bd7f9195","Type":"ContainerStarted","Data":"e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb"}
Jul 13 21:14:14.010868 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:14.010592    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55cc55db57-xjnx5" event={"ID":"cb7f0b85-9a02-4872-b384-0410bd7f9195","Type":"ContainerStarted","Data":"04f3b9e6096d017bf28edec2e19b342427d5511fe43f0b478bd5992a5fd48c55"}
Jul 13 21:14:14.025645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:14.025600    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55cc55db57-xjnx5" podStartSLOduration=2.025587599 podStartE2EDuration="2.025587599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:14:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:14:14.024545618 +0000 UTC m=+161.537295357" watchObservedRunningTime="2026-07-13 21:14:14.025587599 +0000 UTC m=+161.538337338"
Jul 13 21:14:23.112534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:23.112505    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:23.112919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:23.112844    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:23.117139 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:23.117116    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:24.053387 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:24.053360    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:14:24.100652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:24.100620    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8f47fcc7-jzr7h"]
Jul 13 21:14:37.785940 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:37.785886    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6b64b47bfd-z2j2v" podUID="04bfc789-58d1-4cba-bf3a-023b91389581" containerName="console" containerID="cri-o://1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44" gracePeriod=15
Jul 13 21:14:38.031807 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.031786    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:38.105632 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105604    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvp6t\" (UniqueName: \"kubernetes.io/projected/04bfc789-58d1-4cba-bf3a-023b91389581-kube-api-access-hvp6t\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.105804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105647    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-oauth-serving-cert\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.105804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105681    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-console-config\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.105804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105758    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-trusted-ca-bundle\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.105804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105769    2564 generic.go:356] "Generic (PLEG): container finished" podID="04bfc789-58d1-4cba-bf3a-023b91389581" containerID="1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44" exitCode=0
Jul 13 21:14:38.106017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105794    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-service-ca\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.106017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105836    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-serving-cert\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.106017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64b47bfd-z2j2v" event={"ID":"04bfc789-58d1-4cba-bf3a-023b91389581","Type":"ContainerDied","Data":"1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44"}
Jul 13 21:14:38.106017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105890    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6b64b47bfd-z2j2v" event={"ID":"04bfc789-58d1-4cba-bf3a-023b91389581","Type":"ContainerDied","Data":"985e4f4cef28a246ba334e1683103ea3d986d17b577e619a885359f2208d9a5a"}
Jul 13 21:14:38.106017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.105915    2564 scope.go:117] "RemoveContainer" containerID="1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44"
Jul 13 21:14:38.106269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106084    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6b64b47bfd-z2j2v"
Jul 13 21:14:38.106269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106121    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:38.106269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106165    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-console-config" (OuterVolumeSpecName: "console-config") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:38.106269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106219    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-oauth-config\") pod \"04bfc789-58d1-4cba-bf3a-023b91389581\" (UID: \"04bfc789-58d1-4cba-bf3a-023b91389581\") "
Jul 13 21:14:38.106459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106199    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:38.106459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106244    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-service-ca" (OuterVolumeSpecName: "service-ca") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:38.106554 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106522    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-oauth-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.106554 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106537    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-console-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.106554 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106547    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-trusted-ca-bundle\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.106702 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.106560    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04bfc789-58d1-4cba-bf3a-023b91389581-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.108106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.108080    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04bfc789-58d1-4cba-bf3a-023b91389581-kube-api-access-hvp6t" (OuterVolumeSpecName: "kube-api-access-hvp6t") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "kube-api-access-hvp6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:14:38.108270 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.108243    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:14:38.108377 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.108300    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "04bfc789-58d1-4cba-bf3a-023b91389581" (UID: "04bfc789-58d1-4cba-bf3a-023b91389581"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:14:38.125761 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.125736    2564 scope.go:117] "RemoveContainer" containerID="1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44"
Jul 13 21:14:38.126029 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:14:38.126012    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44\": container with ID starting with 1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44 not found: ID does not exist" containerID="1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44"
Jul 13 21:14:38.126111 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.126038    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44"} err="failed to get container status \"1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44\": rpc error: code = NotFound desc = could not find container \"1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44\": container with ID starting with 1fddf37127ab613b5fb39a01cbc859a50c3cd740357146c3de3316038246ac44 not found: ID does not exist"
Jul 13 21:14:38.207655 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.207623    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.207655 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.207650    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04bfc789-58d1-4cba-bf3a-023b91389581-console-oauth-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.207655 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.207660    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvp6t\" (UniqueName: \"kubernetes.io/projected/04bfc789-58d1-4cba-bf3a-023b91389581-kube-api-access-hvp6t\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:38.428164 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.428129    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6b64b47bfd-z2j2v"]
Jul 13 21:14:38.432095 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:38.432072    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6b64b47bfd-z2j2v"]
Jul 13 21:14:39.075635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:39.075602    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04bfc789-58d1-4cba-bf3a-023b91389581" path="/var/lib/kubelet/pods/04bfc789-58d1-4cba-bf3a-023b91389581/volumes"
Jul 13 21:14:49.124941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.124884    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d8f47fcc7-jzr7h" podUID="ec22cfb9-e3ec-4503-a236-347c2058fe9c" containerName="console" containerID="cri-o://b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566" gracePeriod=15
Jul 13 21:14:49.365376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.365354    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:14:49.402256 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402180    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-config\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402256 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402215    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-serving-cert\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402256 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402231    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-oauth-serving-cert\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402256 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402255    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-trusted-ca-bundle\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402549 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402337    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgfkh\" (UniqueName: \"kubernetes.io/projected/ec22cfb9-e3ec-4503-a236-347c2058fe9c-kube-api-access-wgfkh\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402549 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402375    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-service-ca\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402549 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402399    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-oauth-config\") pod \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\" (UID: \"ec22cfb9-e3ec-4503-a236-347c2058fe9c\") "
Jul 13 21:14:49.402695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402575    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-config" (OuterVolumeSpecName: "console-config") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:49.402695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402634    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:49.402695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402686    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:49.402837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402699    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-trusted-ca-bundle\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:49.402954 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402927    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-service-ca" (OuterVolumeSpecName: "service-ca") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:49.403036 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.402967    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:14:49.404619 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.404598    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:14:49.404924 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.404901    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec22cfb9-e3ec-4503-a236-347c2058fe9c-kube-api-access-wgfkh" (OuterVolumeSpecName: "kube-api-access-wgfkh") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "kube-api-access-wgfkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:14:49.404924 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.404910    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ec22cfb9-e3ec-4503-a236-347c2058fe9c" (UID: "ec22cfb9-e3ec-4503-a236-347c2058fe9c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:14:49.503339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.503314    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgfkh\" (UniqueName: \"kubernetes.io/projected/ec22cfb9-e3ec-4503-a236-347c2058fe9c-kube-api-access-wgfkh\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:49.503339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.503339    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:49.503501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.503349    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-oauth-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:49.503501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.503358    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ec22cfb9-e3ec-4503-a236-347c2058fe9c-console-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:49.503501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:49.503367    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ec22cfb9-e3ec-4503-a236-347c2058fe9c-oauth-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:14:50.156117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.156079    2564 generic.go:356] "Generic (PLEG): container finished" podID="ec22cfb9-e3ec-4503-a236-347c2058fe9c" containerID="b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566" exitCode=0
Jul 13 21:14:50.156117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.156119    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d8f47fcc7-jzr7h"
Jul 13 21:14:50.156647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.156160    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8f47fcc7-jzr7h" event={"ID":"ec22cfb9-e3ec-4503-a236-347c2058fe9c","Type":"ContainerDied","Data":"b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566"}
Jul 13 21:14:50.156647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.156212    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d8f47fcc7-jzr7h" event={"ID":"ec22cfb9-e3ec-4503-a236-347c2058fe9c","Type":"ContainerDied","Data":"5f4491eebbde959052df0e5b0911b4623c2174fe22b8f12148bb06405fd70138"}
Jul 13 21:14:50.156647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.156234    2564 scope.go:117] "RemoveContainer" containerID="b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566"
Jul 13 21:14:50.164966 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.164949    2564 scope.go:117] "RemoveContainer" containerID="b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566"
Jul 13 21:14:50.165229 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:14:50.165213    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566\": container with ID starting with b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566 not found: ID does not exist" containerID="b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566"
Jul 13 21:14:50.165281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.165239    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566"} err="failed to get container status \"b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566\": rpc error: code = NotFound desc = could not find container \"b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566\": container with ID starting with b091509866bf77f63729b9992e9ea2a58b70da1ca7ea7aa4343ef22a5f8e8566 not found: ID does not exist"
Jul 13 21:14:50.176637 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.176615    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d8f47fcc7-jzr7h"]
Jul 13 21:14:50.181799 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:50.181781    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d8f47fcc7-jzr7h"]
Jul 13 21:14:51.076635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:14:51.076605    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec22cfb9-e3ec-4503-a236-347c2058fe9c" path="/var/lib/kubelet/pods/ec22cfb9-e3ec-4503-a236-347c2058fe9c/volumes"
Jul 13 21:15:51.387965 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.387927    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fslxv"]
Jul 13 21:15:51.388457 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.388435    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec22cfb9-e3ec-4503-a236-347c2058fe9c" containerName="console"
Jul 13 21:15:51.388457 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.388455    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec22cfb9-e3ec-4503-a236-347c2058fe9c" containerName="console"
Jul 13 21:15:51.388570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.388495    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04bfc789-58d1-4cba-bf3a-023b91389581" containerName="console"
Jul 13 21:15:51.388570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.388507    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="04bfc789-58d1-4cba-bf3a-023b91389581" containerName="console"
Jul 13 21:15:51.388662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.388582    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="04bfc789-58d1-4cba-bf3a-023b91389581" containerName="console"
Jul 13 21:15:51.388662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.388602    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec22cfb9-e3ec-4503-a236-347c2058fe9c" containerName="console"
Jul 13 21:15:51.392750 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.392728    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.395129 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.395107    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-86d8p\""
Jul 13 21:15:51.395209 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.395120    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 13 21:15:51.396409 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.396390    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 13 21:15:51.398093 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.398071    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fslxv"]
Jul 13 21:15:51.515632 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.515589    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e97472a-cea5-45e7-b2e3-707ad5bb80a8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fslxv\" (UID: \"8e97472a-cea5-45e7-b2e3-707ad5bb80a8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.515808 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.515649    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg69v\" (UniqueName: \"kubernetes.io/projected/8e97472a-cea5-45e7-b2e3-707ad5bb80a8-kube-api-access-vg69v\") pod \"cert-manager-webhook-6486ffd997-fslxv\" (UID: \"8e97472a-cea5-45e7-b2e3-707ad5bb80a8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.616293 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.616261    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e97472a-cea5-45e7-b2e3-707ad5bb80a8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fslxv\" (UID: \"8e97472a-cea5-45e7-b2e3-707ad5bb80a8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.616470 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.616303    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg69v\" (UniqueName: \"kubernetes.io/projected/8e97472a-cea5-45e7-b2e3-707ad5bb80a8-kube-api-access-vg69v\") pod \"cert-manager-webhook-6486ffd997-fslxv\" (UID: \"8e97472a-cea5-45e7-b2e3-707ad5bb80a8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.624302 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.624276    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/8e97472a-cea5-45e7-b2e3-707ad5bb80a8-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-fslxv\" (UID: \"8e97472a-cea5-45e7-b2e3-707ad5bb80a8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.624409 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.624320    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg69v\" (UniqueName: \"kubernetes.io/projected/8e97472a-cea5-45e7-b2e3-707ad5bb80a8-kube-api-access-vg69v\") pod \"cert-manager-webhook-6486ffd997-fslxv\" (UID: \"8e97472a-cea5-45e7-b2e3-707ad5bb80a8\") " pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.711947 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.711862    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:15:51.835748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:51.835719    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-fslxv"]
Jul 13 21:15:51.839160 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:15:51.839130    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8e97472a_cea5_45e7_b2e3_707ad5bb80a8.slice/crio-0292093f09da4c9de2f3737f8242bf626919bec4d1eb0c6479ab67a1374c533a WatchSource:0}: Error finding container 0292093f09da4c9de2f3737f8242bf626919bec4d1eb0c6479ab67a1374c533a: Status 404 returned error can't find the container with id 0292093f09da4c9de2f3737f8242bf626919bec4d1eb0c6479ab67a1374c533a
Jul 13 21:15:52.407451 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.407411    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv" event={"ID":"8e97472a-cea5-45e7-b2e3-707ad5bb80a8","Type":"ContainerStarted","Data":"0292093f09da4c9de2f3737f8242bf626919bec4d1eb0c6479ab67a1374c533a"}
Jul 13 21:15:52.774380 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.774303    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"]
Jul 13 21:15:52.777763 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.777739    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:52.780181 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.780153    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-69jwq\""
Jul 13 21:15:52.789701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.789678    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"]
Jul 13 21:15:52.929523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.929485    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e74727ff-eaad-472f-bc11-ee81df66cd8f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8cq4f\" (UID: \"e74727ff-eaad-472f-bc11-ee81df66cd8f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:52.929722 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:52.929537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-94vvg\" (UniqueName: \"kubernetes.io/projected/e74727ff-eaad-472f-bc11-ee81df66cd8f-kube-api-access-94vvg\") pod \"cert-manager-cainjector-559cdd8c87-8cq4f\" (UID: \"e74727ff-eaad-472f-bc11-ee81df66cd8f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:53.030333 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.030244    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e74727ff-eaad-472f-bc11-ee81df66cd8f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8cq4f\" (UID: \"e74727ff-eaad-472f-bc11-ee81df66cd8f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:53.030333 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.030291    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-94vvg\" (UniqueName: \"kubernetes.io/projected/e74727ff-eaad-472f-bc11-ee81df66cd8f-kube-api-access-94vvg\") pod \"cert-manager-cainjector-559cdd8c87-8cq4f\" (UID: \"e74727ff-eaad-472f-bc11-ee81df66cd8f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:53.038819 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.038788    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-94vvg\" (UniqueName: \"kubernetes.io/projected/e74727ff-eaad-472f-bc11-ee81df66cd8f-kube-api-access-94vvg\") pod \"cert-manager-cainjector-559cdd8c87-8cq4f\" (UID: \"e74727ff-eaad-472f-bc11-ee81df66cd8f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:53.039108 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.039076    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e74727ff-eaad-472f-bc11-ee81df66cd8f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8cq4f\" (UID: \"e74727ff-eaad-472f-bc11-ee81df66cd8f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:53.089887 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.089859    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"
Jul 13 21:15:53.240381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.240356    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f"]
Jul 13 21:15:53.242381 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:15:53.242352    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode74727ff_eaad_472f_bc11_ee81df66cd8f.slice/crio-cfb03e73e8b72483564526f5d1bffc6d26787c65c2c3079c74b8d7d6cfe6a1db WatchSource:0}: Error finding container cfb03e73e8b72483564526f5d1bffc6d26787c65c2c3079c74b8d7d6cfe6a1db: Status 404 returned error can't find the container with id cfb03e73e8b72483564526f5d1bffc6d26787c65c2c3079c74b8d7d6cfe6a1db
Jul 13 21:15:53.413938 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:53.413893    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f" event={"ID":"e74727ff-eaad-472f-bc11-ee81df66cd8f","Type":"ContainerStarted","Data":"cfb03e73e8b72483564526f5d1bffc6d26787c65c2c3079c74b8d7d6cfe6a1db"}
Jul 13 21:15:56.429876 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:56.429837    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f" event={"ID":"e74727ff-eaad-472f-bc11-ee81df66cd8f","Type":"ContainerStarted","Data":"38aea850a16aa7169b044391a592390fd6960681834575f59a2d4cc1624496f2"}
Jul 13 21:15:56.445393 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:15:56.445352    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8cq4f" podStartSLOduration=1.355211624 podStartE2EDuration="4.445338108s" totalImagesPullingTime="3.090126484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:15:56.443292537 +0000 UTC m=+263.956042283" watchObservedRunningTime="2026-07-13 21:15:56.445338108 +0000 UTC m=+263.958087826"
Jul 13 21:16:06.477140 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:06.477104    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv" event={"ID":"8e97472a-cea5-45e7-b2e3-707ad5bb80a8","Type":"ContainerStarted","Data":"4a35e6f561cee381cd436a3b282055597baa6ba8a6a5bd4c2df960c71f0fc810"}
Jul 13 21:16:06.477521 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:06.477207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:16:06.495480 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:06.495415    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv" podStartSLOduration=1.065881241 podStartE2EDuration="15.495396553s" totalImagesPullingTime="14.429515312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:15:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:06.493849451 +0000 UTC m=+274.006599193" watchObservedRunningTime="2026-07-13 21:16:06.495396553 +0000 UTC m=+274.008146293"
Jul 13 21:16:12.483675 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:12.483645    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-fslxv"
Jul 13 21:16:23.803483 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.803412    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"]
Jul 13 21:16:23.805740 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.805725    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.809438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.809420    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 13 21:16:23.809671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.809653    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 13 21:16:23.809780 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.809677    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 13 21:16:23.810134 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.810121    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 13 21:16:23.810134 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.810127    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q5mf8\""
Jul 13 21:16:23.825496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.825466    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"]
Jul 13 21:16:23.896448 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.896416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67458df2-a147-4ca6-97d1-c4b22a2803e4-manifests\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.896648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.896474    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67458df2-a147-4ca6-97d1-c4b22a2803e4-tmp\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.896648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.896515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67458df2-a147-4ca6-97d1-c4b22a2803e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.896648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.896552    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67458df2-a147-4ca6-97d1-c4b22a2803e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.896648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.896611    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7454\" (UniqueName: \"kubernetes.io/projected/67458df2-a147-4ca6-97d1-c4b22a2803e4-kube-api-access-v7454\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.997771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.997741    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67458df2-a147-4ca6-97d1-c4b22a2803e4-tmp\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.997771 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.997773    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67458df2-a147-4ca6-97d1-c4b22a2803e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.997994 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.997796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67458df2-a147-4ca6-97d1-c4b22a2803e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.997994 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.997829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7454\" (UniqueName: \"kubernetes.io/projected/67458df2-a147-4ca6-97d1-c4b22a2803e4-kube-api-access-v7454\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.998132 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.998002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67458df2-a147-4ca6-97d1-c4b22a2803e4-manifests\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:23.998345 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:23.998315    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/67458df2-a147-4ca6-97d1-c4b22a2803e4-manifests\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:24.000221 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.000195    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/67458df2-a147-4ca6-97d1-c4b22a2803e4-tmp\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:24.000299 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.000280    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/67458df2-a147-4ca6-97d1-c4b22a2803e4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:24.000299 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.000287    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/67458df2-a147-4ca6-97d1-c4b22a2803e4-webhook-cert\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:24.006185 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.006162    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7454\" (UniqueName: \"kubernetes.io/projected/67458df2-a147-4ca6-97d1-c4b22a2803e4-kube-api-access-v7454\") pod \"opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf\" (UID: \"67458df2-a147-4ca6-97d1-c4b22a2803e4\") " pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:24.115289 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.115257    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:24.250198 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.250173    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"]
Jul 13 21:16:24.252827 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:24.252796    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67458df2_a147_4ca6_97d1_c4b22a2803e4.slice/crio-2d089efb15a128352e54b9befa9d8211daacd247e86400bcf92d310a142a5143 WatchSource:0}: Error finding container 2d089efb15a128352e54b9befa9d8211daacd247e86400bcf92d310a142a5143: Status 404 returned error can't find the container with id 2d089efb15a128352e54b9befa9d8211daacd247e86400bcf92d310a142a5143
Jul 13 21:16:24.553707 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:24.553604    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf" event={"ID":"67458df2-a147-4ca6-97d1-c4b22a2803e4","Type":"ContainerStarted","Data":"2d089efb15a128352e54b9befa9d8211daacd247e86400bcf92d310a142a5143"}
Jul 13 21:16:27.573206 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:27.573167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf" event={"ID":"67458df2-a147-4ca6-97d1-c4b22a2803e4","Type":"ContainerStarted","Data":"1b66eeb2c94fe808da7a1996dc48da2dffc12e11a29a27155a97aa10567885fe"}
Jul 13 21:16:29.583121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:29.583082    2564 generic.go:356] "Generic (PLEG): container finished" podID="67458df2-a147-4ca6-97d1-c4b22a2803e4" containerID="1b66eeb2c94fe808da7a1996dc48da2dffc12e11a29a27155a97aa10567885fe" exitCode=0
Jul 13 21:16:29.583489 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:29.583159    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf" event={"ID":"67458df2-a147-4ca6-97d1-c4b22a2803e4","Type":"ContainerDied","Data":"1b66eeb2c94fe808da7a1996dc48da2dffc12e11a29a27155a97aa10567885fe"}
Jul 13 21:16:30.591494 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:30.591445    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf" event={"ID":"67458df2-a147-4ca6-97d1-c4b22a2803e4","Type":"ContainerStarted","Data":"c4e338b1f1c94973812d9782af10598a86435e929ac0313ed6bc51b07072667a"}
Jul 13 21:16:30.592553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:30.592514    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:30.616042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:30.615991    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf" podStartSLOduration=2.909817453 podStartE2EDuration="7.615974542s" totalImagesPullingTime="3.619572915s" totalInitContainerRuntime="1.086584174s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:30.613475075 +0000 UTC m=+298.126224848" watchObservedRunningTime="2026-07-13 21:16:30.615974542 +0000 UTC m=+298.128724283"
Jul 13 21:16:32.934902 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:32.934876    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:16:32.935337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:32.935249    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:16:32.941665 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:32.941641    2564 kubelet.go:1648] "Image garbage collection succeeded"
Jul 13 21:16:41.597795 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:41.597769    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf"
Jul 13 21:16:44.704750 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.704713    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6wpx8"]
Jul 13 21:16:44.707962 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.707936    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:44.712125 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.712100    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 13 21:16:44.714612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.714592    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-cx2fm\""
Jul 13 21:16:44.722090 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.722047    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6wpx8"]
Jul 13 21:16:44.788746 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.788718    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b5bee33-069b-499a-b32f-226173b22122-cert\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:44.788919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.788803    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srdl4\" (UniqueName: \"kubernetes.io/projected/5b5bee33-069b-499a-b32f-226173b22122-kube-api-access-srdl4\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:44.889829 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.889790    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b5bee33-069b-499a-b32f-226173b22122-cert\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:44.890022 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.889876    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srdl4\" (UniqueName: \"kubernetes.io/projected/5b5bee33-069b-499a-b32f-226173b22122-kube-api-access-srdl4\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:44.890022 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:16:44.889942    2564 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 13 21:16:44.890022 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:16:44.890008    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5b5bee33-069b-499a-b32f-226173b22122-cert podName:5b5bee33-069b-499a-b32f-226173b22122 nodeName:}" failed. No retries permitted until 2026-07-13 21:16:45.389989913 +0000 UTC m=+312.902739635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5b5bee33-069b-499a-b32f-226173b22122-cert") pod "odh-model-controller-7998c67dc4-6wpx8" (UID: "5b5bee33-069b-499a-b32f-226173b22122") : secret "odh-model-controller-webhook-cert" not found
Jul 13 21:16:44.932885 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:44.932848    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srdl4\" (UniqueName: \"kubernetes.io/projected/5b5bee33-069b-499a-b32f-226173b22122-kube-api-access-srdl4\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:45.344683 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.344650    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"]
Jul 13 21:16:45.347515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.347496    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.350281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.350259    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 13 21:16:45.350281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.350268    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 13 21:16:45.350412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.350357    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 13 21:16:45.350412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.350385    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-qlz4z\""
Jul 13 21:16:45.351254 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.351237    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 13 21:16:45.359727 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.359703    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"]
Jul 13 21:16:45.395763 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.395736    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b5bee33-069b-499a-b32f-226173b22122-cert\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:45.398048 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.398020    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5b5bee33-069b-499a-b32f-226173b22122-cert\") pod \"odh-model-controller-7998c67dc4-6wpx8\" (UID: \"5b5bee33-069b-499a-b32f-226173b22122\") " pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:45.496942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.496904    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63af4ef4-2065-4c23-a017-d5db54b5c4e9-tls-certs\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.497160 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.497012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63af4ef4-2065-4c23-a017-d5db54b5c4e9-tmp\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.497160 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.497076    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jf8k8\" (UniqueName: \"kubernetes.io/projected/63af4ef4-2065-4c23-a017-d5db54b5c4e9-kube-api-access-jf8k8\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.598383 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.598296    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63af4ef4-2065-4c23-a017-d5db54b5c4e9-tmp\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.598383 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.598354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jf8k8\" (UniqueName: \"kubernetes.io/projected/63af4ef4-2065-4c23-a017-d5db54b5c4e9-kube-api-access-jf8k8\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.598612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.598462    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63af4ef4-2065-4c23-a017-d5db54b5c4e9-tls-certs\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.600883 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.600846    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/63af4ef4-2065-4c23-a017-d5db54b5c4e9-tmp\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.601119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.601100    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63af4ef4-2065-4c23-a017-d5db54b5c4e9-tls-certs\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.611731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.611709    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jf8k8\" (UniqueName: \"kubernetes.io/projected/63af4ef4-2065-4c23-a017-d5db54b5c4e9-kube-api-access-jf8k8\") pod \"kube-auth-proxy-7cf89dc8f8-hpc52\" (UID: \"63af4ef4-2065-4c23-a017-d5db54b5c4e9\") " pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.618634 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.618600    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:45.658199 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.658167    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"
Jul 13 21:16:45.785362 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.785329    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qrgqf"]
Jul 13 21:16:45.789395 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.789368    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:45.792196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.792177    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 13 21:16:45.798971 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.798947    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qrgqf"]
Jul 13 21:16:45.801646 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.801550    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-6wpx8"]
Jul 13 21:16:45.804487 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:45.804462    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b5bee33_069b_499a_b32f_226173b22122.slice/crio-227699d61b38b7e29d860bac8809f1b37e23323e598d0a5220e9895078b055ad WatchSource:0}: Error finding container 227699d61b38b7e29d860bac8809f1b37e23323e598d0a5220e9895078b055ad: Status 404 returned error can't find the container with id 227699d61b38b7e29d860bac8809f1b37e23323e598d0a5220e9895078b055ad
Jul 13 21:16:45.805913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.805894    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:16:45.814105 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.814044    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52"]
Jul 13 21:16:45.816483 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:45.816461    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63af4ef4_2065_4c23_a017_d5db54b5c4e9.slice/crio-87239337a4574a8a3c95b58343073aad5d291f7473306b463eb4c5ad122e7464 WatchSource:0}: Error finding container 87239337a4574a8a3c95b58343073aad5d291f7473306b463eb4c5ad122e7464: Status 404 returned error can't find the container with id 87239337a4574a8a3c95b58343073aad5d291f7473306b463eb4c5ad122e7464
Jul 13 21:16:45.904493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.904410    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65d3467c-8fd5-4531-a2d8-619d39c47bcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-qrgqf\" (UID: \"65d3467c-8fd5-4531-a2d8-619d39c47bcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:45.904493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:45.904441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5n9mf\" (UniqueName: \"kubernetes.io/projected/65d3467c-8fd5-4531-a2d8-619d39c47bcc-kube-api-access-5n9mf\") pod \"model-serving-api-54d89bd5fc-qrgqf\" (UID: \"65d3467c-8fd5-4531-a2d8-619d39c47bcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:46.005075 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.005031    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65d3467c-8fd5-4531-a2d8-619d39c47bcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-qrgqf\" (UID: \"65d3467c-8fd5-4531-a2d8-619d39c47bcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:46.005248 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.005098    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5n9mf\" (UniqueName: \"kubernetes.io/projected/65d3467c-8fd5-4531-a2d8-619d39c47bcc-kube-api-access-5n9mf\") pod \"model-serving-api-54d89bd5fc-qrgqf\" (UID: \"65d3467c-8fd5-4531-a2d8-619d39c47bcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:46.007458 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.007432    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/65d3467c-8fd5-4531-a2d8-619d39c47bcc-tls-certs\") pod \"model-serving-api-54d89bd5fc-qrgqf\" (UID: \"65d3467c-8fd5-4531-a2d8-619d39c47bcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:46.014268 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.014244    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5n9mf\" (UniqueName: \"kubernetes.io/projected/65d3467c-8fd5-4531-a2d8-619d39c47bcc-kube-api-access-5n9mf\") pod \"model-serving-api-54d89bd5fc-qrgqf\" (UID: \"65d3467c-8fd5-4531-a2d8-619d39c47bcc\") " pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:46.102250 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.102214    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:46.236521 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.236478    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-qrgqf"]
Jul 13 21:16:46.240301 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:46.240278    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65d3467c_8fd5_4531_a2d8_619d39c47bcc.slice/crio-47308667882db1ed145d2b717d518f63dc60abbffceb1694bd37f9f60b9a823c WatchSource:0}: Error finding container 47308667882db1ed145d2b717d518f63dc60abbffceb1694bd37f9f60b9a823c: Status 404 returned error can't find the container with id 47308667882db1ed145d2b717d518f63dc60abbffceb1694bd37f9f60b9a823c
Jul 13 21:16:46.662101 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.662049    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf" event={"ID":"65d3467c-8fd5-4531-a2d8-619d39c47bcc","Type":"ContainerStarted","Data":"47308667882db1ed145d2b717d518f63dc60abbffceb1694bd37f9f60b9a823c"}
Jul 13 21:16:46.663758 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.663730    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8" event={"ID":"5b5bee33-069b-499a-b32f-226173b22122","Type":"ContainerStarted","Data":"227699d61b38b7e29d860bac8809f1b37e23323e598d0a5220e9895078b055ad"}
Jul 13 21:16:46.665153 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.665129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52" event={"ID":"63af4ef4-2065-4c23-a017-d5db54b5c4e9","Type":"ContainerStarted","Data":"87239337a4574a8a3c95b58343073aad5d291f7473306b463eb4c5ad122e7464"}
Jul 13 21:16:46.672224 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.672191    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"]
Jul 13 21:16:46.678075 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.678037    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.681921 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.681590    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 13 21:16:46.682108 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.681956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 13 21:16:46.682108 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.681982    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-nhhz5\""
Jul 13 21:16:46.682238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.681956    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 13 21:16:46.682292 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.682244    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 13 21:16:46.694777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.694753    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"]
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.813922    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.814021    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.814050    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.814104    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.814193    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.814218    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fc46z\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-kube-api-access-fc46z\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.814382 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.814253    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915300    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915346    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fc46z\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-kube-api-access-fc46z\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915611 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915480    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915611 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915532    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915720 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915613    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915720 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.915720 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.915677    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.916325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.916299    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.918598 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.918553    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.918716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.918673    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.918817 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.918790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-local-certs\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.919315 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.919293    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-cacerts\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.924941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.924896    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fc46z\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-kube-api-access-fc46z\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.925184 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.925107    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-token\") pod \"istiod-openshift-gateway-7574459656-wt5gk\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:46.994363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:46.994277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:47.176539 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:47.175611    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"]
Jul 13 21:16:47.178531 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:47.178497    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2148bbf_1f32_4c64_91c5_0b75e65b08c0.slice/crio-ecece601735e85115637270934763376e54b7f7a33271aa61924be6490ca5c24 WatchSource:0}: Error finding container ecece601735e85115637270934763376e54b7f7a33271aa61924be6490ca5c24: Status 404 returned error can't find the container with id ecece601735e85115637270934763376e54b7f7a33271aa61924be6490ca5c24
Jul 13 21:16:47.673525 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:47.673490    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" event={"ID":"e2148bbf-1f32-4c64-91c5-0b75e65b08c0","Type":"ContainerStarted","Data":"ecece601735e85115637270934763376e54b7f7a33271aa61924be6490ca5c24"}
Jul 13 21:16:50.693308 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.693271    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf" event={"ID":"65d3467c-8fd5-4531-a2d8-619d39c47bcc","Type":"ContainerStarted","Data":"2fe507b97b3ef3325fadfe87ae61562089fa2f8ab5006a9a1d41a72e3a55569f"}
Jul 13 21:16:50.693744 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.693344    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:16:50.695049 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.694986    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8" event={"ID":"5b5bee33-069b-499a-b32f-226173b22122","Type":"ContainerStarted","Data":"546ac6e64a02786c3a7aa911667814cbfdddf5328625121692a48b24c6336178"}
Jul 13 21:16:50.695224 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.695081    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:16:50.697504 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.697471    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52" event={"ID":"63af4ef4-2065-4c23-a017-d5db54b5c4e9","Type":"ContainerStarted","Data":"27567b58211539d3ab8e9c8bdb8faa4bfeb07a1fefb417d96a949c5c76b4c334"}
Jul 13 21:16:50.713452 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.713394    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf" podStartSLOduration=1.9225065 podStartE2EDuration="5.713373144s" totalImagesPullingTime="3.790866644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:50.70969844 +0000 UTC m=+318.222448176" watchObservedRunningTime="2026-07-13 21:16:50.713373144 +0000 UTC m=+318.226122885"
Jul 13 21:16:50.728339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.728276    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8" podStartSLOduration=2.5013860919999997 podStartE2EDuration="6.728253996s" totalImagesPullingTime="4.226867904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:50.726233956 +0000 UTC m=+318.238983693" watchObservedRunningTime="2026-07-13 21:16:50.728253996 +0000 UTC m=+318.241003739"
Jul 13 21:16:50.745341 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:50.742868    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7cf89dc8f8-hpc52" podStartSLOduration=1.522035458 podStartE2EDuration="5.742848369s" totalImagesPullingTime="4.220812911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:50.741927711 +0000 UTC m=+318.254677453" watchObservedRunningTime="2026-07-13 21:16:50.742848369 +0000 UTC m=+318.255598113"
Jul 13 21:16:51.467381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:51.467337    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:16:51.467469 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:51.467407    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:16:51.704533 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:51.704480    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" event={"ID":"e2148bbf-1f32-4c64-91c5-0b75e65b08c0","Type":"ContainerStarted","Data":"3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2"}
Jul 13 21:16:51.704993 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:51.704972    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:51.766769 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:51.766658    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" podStartSLOduration=1.48087345 podStartE2EDuration="5.766635897s" totalImagesPullingTime="4.285762447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:51.766103991 +0000 UTC m=+319.278853733" watchObservedRunningTime="2026-07-13 21:16:51.766635897 +0000 UTC m=+319.279385637"
Jul 13 21:16:52.698832 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.698795    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"]
Jul 13 21:16:52.702636 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.702615    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.705538 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.705515    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-lfctx\""
Jul 13 21:16:52.711134 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.711109    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-wt5gk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 21:16:52.711215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.711163    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:16:52.719399 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.719378    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"]
Jul 13 21:16:52.876457 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876422    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.876613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876537    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.876690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876641    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.876742 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.876813 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jh9lg\" (UniqueName: \"kubernetes.io/projected/94dbfa33-448a-4b9d-987f-636cbe6e3947-kube-api-access-jh9lg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.876904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876870    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.877005 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.876982    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.877151 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.877014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.877151 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.877087    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94dbfa33-448a-4b9d-987f-636cbe6e3947-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.978542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.978457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.978542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.978502    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.978542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.978535    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94dbfa33-448a-4b9d-987f-636cbe6e3947-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.978805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.978668    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.978805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.978770    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.978805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.978799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979426 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979400    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/94dbfa33-448a-4b9d-987f-636cbe6e3947-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979436    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979098    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979521    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jh9lg\" (UniqueName: \"kubernetes.io/projected/94dbfa33-448a-4b9d-987f-636cbe6e3947-kube-api-access-jh9lg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.979976 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.979873    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.981717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.981664    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.981975 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.981957    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:52.998557 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:52.998533    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jh9lg\" (UniqueName: \"kubernetes.io/projected/94dbfa33-448a-4b9d-987f-636cbe6e3947-kube-api-access-jh9lg\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:53.001023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:53.000994    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/94dbfa33-448a-4b9d-987f-636cbe6e3947-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4z5b55\" (UID: \"94dbfa33-448a-4b9d-987f-636cbe6e3947\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:53.013966 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:53.013939    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:53.173296 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:53.173273    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"]
Jul 13 21:16:53.175451 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:53.175424    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94dbfa33_448a_4b9d_987f_636cbe6e3947.slice/crio-154dbc4f761f0e0e89b80faa1cc52c3c97bc2fac38f2e841a9ae861d6985c7b7 WatchSource:0}: Error finding container 154dbc4f761f0e0e89b80faa1cc52c3c97bc2fac38f2e841a9ae861d6985c7b7: Status 404 returned error can't find the container with id 154dbc4f761f0e0e89b80faa1cc52c3c97bc2fac38f2e841a9ae861d6985c7b7
Jul 13 21:16:53.715154 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:53.715119    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55" event={"ID":"94dbfa33-448a-4b9d-987f-636cbe6e3947","Type":"ContainerStarted","Data":"154dbc4f761f0e0e89b80faa1cc52c3c97bc2fac38f2e841a9ae861d6985c7b7"}
Jul 13 21:16:53.715945 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:53.715921    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:55.184515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.184484    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"]
Jul 13 21:16:55.191868 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.191847    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"]
Jul 13 21:16:55.191993 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.191978    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.194431 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.194403    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 13 21:16:55.194612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.194593    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-r97zp\""
Jul 13 21:16:55.195277 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.195260    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.197736 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.197717    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-mpjvl\""
Jul 13 21:16:55.197951 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.197932    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 13 21:16:55.206661 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.205317    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"]
Jul 13 21:16:55.207836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.207797    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"]
Jul 13 21:16:55.307117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.307039    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wwnh\" (UniqueName: \"kubernetes.io/projected/eba5395b-ca42-4a9d-a331-3d7bdebb4708-kube-api-access-9wwnh\") pod \"llmisvc-controller-manager-777c6bd7d5-982vc\" (UID: \"eba5395b-ca42-4a9d-a331-3d7bdebb4708\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.307275 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.307178    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eba5395b-ca42-4a9d-a331-3d7bdebb4708-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-982vc\" (UID: \"eba5395b-ca42-4a9d-a331-3d7bdebb4708\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.307441 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.307423    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26316338-bc6d-43e8-b66f-55e96fda2f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.307525 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.307457    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kld67\" (UniqueName: \"kubernetes.io/projected/26316338-bc6d-43e8-b66f-55e96fda2f6c-kube-api-access-kld67\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.408383 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.408354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wwnh\" (UniqueName: \"kubernetes.io/projected/eba5395b-ca42-4a9d-a331-3d7bdebb4708-kube-api-access-9wwnh\") pod \"llmisvc-controller-manager-777c6bd7d5-982vc\" (UID: \"eba5395b-ca42-4a9d-a331-3d7bdebb4708\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.408558 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.408417    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eba5395b-ca42-4a9d-a331-3d7bdebb4708-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-982vc\" (UID: \"eba5395b-ca42-4a9d-a331-3d7bdebb4708\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.408558 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.408496    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26316338-bc6d-43e8-b66f-55e96fda2f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.408558 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.408524    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kld67\" (UniqueName: \"kubernetes.io/projected/26316338-bc6d-43e8-b66f-55e96fda2f6c-kube-api-access-kld67\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.408735 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:16:55.408650    2564 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 13 21:16:55.408789 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:16:55.408740    2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/26316338-bc6d-43e8-b66f-55e96fda2f6c-cert podName:26316338-bc6d-43e8-b66f-55e96fda2f6c nodeName:}" failed. No retries permitted until 2026-07-13 21:16:55.90871623 +0000 UTC m=+323.421465958 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/26316338-bc6d-43e8-b66f-55e96fda2f6c-cert") pod "kserve-controller-manager-d6f4b65b8-7cq8m" (UID: "26316338-bc6d-43e8-b66f-55e96fda2f6c") : secret "kserve-webhook-server-cert" not found
Jul 13 21:16:55.411187 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.411163    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/eba5395b-ca42-4a9d-a331-3d7bdebb4708-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-982vc\" (UID: \"eba5395b-ca42-4a9d-a331-3d7bdebb4708\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.422182 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.422156    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kld67\" (UniqueName: \"kubernetes.io/projected/26316338-bc6d-43e8-b66f-55e96fda2f6c-kube-api-access-kld67\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.424113 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.424092    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wwnh\" (UniqueName: \"kubernetes.io/projected/eba5395b-ca42-4a9d-a331-3d7bdebb4708-kube-api-access-9wwnh\") pod \"llmisvc-controller-manager-777c6bd7d5-982vc\" (UID: \"eba5395b-ca42-4a9d-a331-3d7bdebb4708\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.524347 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.524263    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:55.846523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.846485    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:16:55.846647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.846572    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:16:55.846647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.846619    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:16:55.913391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.913361    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26316338-bc6d-43e8-b66f-55e96fda2f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.915595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.915573    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"]
Jul 13 21:16:55.915702 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:55.915686    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26316338-bc6d-43e8-b66f-55e96fda2f6c-cert\") pod \"kserve-controller-manager-d6f4b65b8-7cq8m\" (UID: \"26316338-bc6d-43e8-b66f-55e96fda2f6c\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:55.916723 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:55.916696    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podeba5395b_ca42_4a9d_a331_3d7bdebb4708.slice/crio-cb8303b050a2e951edc3beb2aff0a012c38faddc9c023671e4517c465ba9b823 WatchSource:0}: Error finding container cb8303b050a2e951edc3beb2aff0a012c38faddc9c023671e4517c465ba9b823: Status 404 returned error can't find the container with id cb8303b050a2e951edc3beb2aff0a012c38faddc9c023671e4517c465ba9b823
Jul 13 21:16:56.110935 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.110908    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:56.242778 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.242755    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"]
Jul 13 21:16:56.244876 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:16:56.244851    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26316338_bc6d_43e8_b66f_55e96fda2f6c.slice/crio-75d60d5204606999e622328aff4cc443c2e304346ad4d8b958a75177a08058e3 WatchSource:0}: Error finding container 75d60d5204606999e622328aff4cc443c2e304346ad4d8b958a75177a08058e3: Status 404 returned error can't find the container with id 75d60d5204606999e622328aff4cc443c2e304346ad4d8b958a75177a08058e3
Jul 13 21:16:56.687595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.687559    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"]
Jul 13 21:16:56.687882 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.687790    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerName="discovery" containerID="cri-o://3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2" gracePeriod=30
Jul 13 21:16:56.716173 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.716139    2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7574459656-wt5gk container/discovery namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.30:8080/ready\": dial tcp 10.134.0.30:8080: connect: connection refused" start-of-body=
Jul 13 21:16:56.716337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.716214    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerName="discovery" probeResult="failure" output="Get \"http://10.134.0.30:8080/ready\": dial tcp 10.134.0.30:8080: connect: connection refused"
Jul 13 21:16:56.732923 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.732891    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc" event={"ID":"eba5395b-ca42-4a9d-a331-3d7bdebb4708","Type":"ContainerStarted","Data":"cb8303b050a2e951edc3beb2aff0a012c38faddc9c023671e4517c465ba9b823"}
Jul 13 21:16:56.733920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.733897    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m" event={"ID":"26316338-bc6d-43e8-b66f-55e96fda2f6c","Type":"ContainerStarted","Data":"75d60d5204606999e622328aff4cc443c2e304346ad4d8b958a75177a08058e3"}
Jul 13 21:16:56.735124 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.735106    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55" event={"ID":"94dbfa33-448a-4b9d-987f-636cbe6e3947","Type":"ContainerStarted","Data":"ada3d6b32fe5171b0a2a8cc3ea117f71cb1599a47593567e508fee1dd84d3fe6"}
Jul 13 21:16:56.761343 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:56.761293    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55" podStartSLOduration=2.0924748380000002 podStartE2EDuration="4.761275152s" totalImagesPullingTime="2.668800314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:56.759580369 +0000 UTC m=+324.272330113" watchObservedRunningTime="2026-07-13 21:16:56.761275152 +0000 UTC m=+324.274024892"
Jul 13 21:16:57.015034 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.014896    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:57.021415 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.021388    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:57.106565 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.106533    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:57.126363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126329    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-token\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.126553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126432    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-dns-cert\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.126553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126505    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-cacerts\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.126553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126536    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fc46z\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-kube-api-access-fc46z\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.126731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126593    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-local-certs\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.126731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126615    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-kubeconfig\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.126731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.126657    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-ca-configmap\") pod \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\" (UID: \"e2148bbf-1f32-4c64-91c5-0b75e65b08c0\") "
Jul 13 21:16:57.127351 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.127308    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:16:57.129286 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.129248    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-token" (OuterVolumeSpecName: "istio-token") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:16:57.129820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.129796    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:16:57.131316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.131198    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:16:57.131432 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.131389    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-local-certs" (OuterVolumeSpecName: "local-certs") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:16:57.131677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.131642    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-cacerts" (OuterVolumeSpecName: "cacerts") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:16:57.131776 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.131704    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-kube-api-access-fc46z" (OuterVolumeSpecName: "kube-api-access-fc46z") pod "e2148bbf-1f32-4c64-91c5-0b75e65b08c0" (UID: "e2148bbf-1f32-4c64-91c5-0b75e65b08c0"). InnerVolumeSpecName "kube-api-access-fc46z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227512    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-dns-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227542    2564 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-cacerts\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227559    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fc46z\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-kube-api-access-fc46z\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227574    2564 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-local-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227588    2564 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-kubeconfig\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227600    2564 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-csr-ca-configmap\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.227651 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.227612    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2148bbf-1f32-4c64-91c5-0b75e65b08c0-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:16:57.742689 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.742650    2564 generic.go:356] "Generic (PLEG): container finished" podID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerID="3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2" exitCode=0
Jul 13 21:16:57.743191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.742748    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" event={"ID":"e2148bbf-1f32-4c64-91c5-0b75e65b08c0","Type":"ContainerDied","Data":"3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2"}
Jul 13 21:16:57.743191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.742873    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk" event={"ID":"e2148bbf-1f32-4c64-91c5-0b75e65b08c0","Type":"ContainerDied","Data":"ecece601735e85115637270934763376e54b7f7a33271aa61924be6490ca5c24"}
Jul 13 21:16:57.743191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.742905    2564 scope.go:117] "RemoveContainer" containerID="3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2"
Jul 13 21:16:57.743191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.743093    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:57.743396 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.743343    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"
Jul 13 21:16:57.744447 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.744416    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4z5b55"
Jul 13 21:16:57.754199 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.754174    2564 scope.go:117] "RemoveContainer" containerID="3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2"
Jul 13 21:16:57.754517 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:16:57.754484    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2\": container with ID starting with 3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2 not found: ID does not exist" containerID="3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2"
Jul 13 21:16:57.754601 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.754518    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2"} err="failed to get container status \"3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2\": rpc error: code = NotFound desc = could not find container \"3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2\": container with ID starting with 3f1bec6aca64cc9f980c3b60794770aeb93fa64b926d3c240ae3ece96c2f3dc2 not found: ID does not exist"
Jul 13 21:16:57.793738 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.793468    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"]
Jul 13 21:16:57.795820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:57.795788    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-wt5gk"]
Jul 13 21:16:59.078349 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.078311    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" path="/var/lib/kubelet/pods/e2148bbf-1f32-4c64-91c5-0b75e65b08c0/volumes"
Jul 13 21:16:59.756222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.756184    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m" event={"ID":"26316338-bc6d-43e8-b66f-55e96fda2f6c","Type":"ContainerStarted","Data":"f55a952ef254d4aa166711032664b920500eefe5738c0a286f656169bf01fbaf"}
Jul 13 21:16:59.756422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.756288    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:16:59.757735 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.757711    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc" event={"ID":"eba5395b-ca42-4a9d-a331-3d7bdebb4708","Type":"ContainerStarted","Data":"8d043503929b31738d9702657eb566bd2572d5afb7e3e3731ce050b5aff41921"}
Jul 13 21:16:59.757857 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.757845    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:16:59.791226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.791120    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m" podStartSLOduration=1.512283021 podStartE2EDuration="4.79109843s" totalImagesPullingTime="3.278815409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:59.773374909 +0000 UTC m=+327.286124663" watchObservedRunningTime="2026-07-13 21:16:59.79109843 +0000 UTC m=+327.303848166"
Jul 13 21:16:59.792760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:16:59.792731    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc" podStartSLOduration=1.181277687 podStartE2EDuration="4.79272084s" totalImagesPullingTime="3.611443153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:16:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:16:59.790413772 +0000 UTC m=+327.303163512" watchObservedRunningTime="2026-07-13 21:16:59.79272084 +0000 UTC m=+327.305470579"
Jul 13 21:17:01.706876 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:01.706844    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-6wpx8"
Jul 13 21:17:01.709571 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:01.709551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-qrgqf"
Jul 13 21:17:30.768041 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:30.768002    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-982vc"
Jul 13 21:17:30.770840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:30.770818    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7cq8m"
Jul 13 21:17:45.138312 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.138274    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zs7nf"]
Jul 13 21:17:45.138715 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.138701    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerName="discovery"
Jul 13 21:17:45.138758 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.138715    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerName="discovery"
Jul 13 21:17:45.138792 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.138787    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2148bbf-1f32-4c64-91c5-0b75e65b08c0" containerName="discovery"
Jul 13 21:17:45.145188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.145158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:45.147548 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.147517    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 13 21:17:45.147681 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.147550    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-rdzgp\""
Jul 13 21:17:45.148548 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.148529    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 13 21:17:45.155889 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.155864    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zs7nf"]
Jul 13 21:17:45.262728 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.262695    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9g5k\" (UniqueName: \"kubernetes.io/projected/59c1d340-3c32-4c4d-bed1-a6cc90907de7-kube-api-access-q9g5k\") pod \"kuadrant-operator-catalog-zs7nf\" (UID: \"59c1d340-3c32-4c4d-bed1-a6cc90907de7\") " pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:45.364022 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.363985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9g5k\" (UniqueName: \"kubernetes.io/projected/59c1d340-3c32-4c4d-bed1-a6cc90907de7-kube-api-access-q9g5k\") pod \"kuadrant-operator-catalog-zs7nf\" (UID: \"59c1d340-3c32-4c4d-bed1-a6cc90907de7\") " pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:45.371772 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.371746    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9g5k\" (UniqueName: \"kubernetes.io/projected/59c1d340-3c32-4c4d-bed1-a6cc90907de7-kube-api-access-q9g5k\") pod \"kuadrant-operator-catalog-zs7nf\" (UID: \"59c1d340-3c32-4c4d-bed1-a6cc90907de7\") " pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:45.457222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.457133    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:45.500888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.500853    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zs7nf"]
Jul 13 21:17:45.587668 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.587646    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zs7nf"]
Jul 13 21:17:45.590071 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:17:45.590027    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59c1d340_3c32_4c4d_bed1_a6cc90907de7.slice/crio-d2bfafd0d947e4143f5321e9845abb1c3753b540a3d050c78f32efda3ab5bab8 WatchSource:0}: Error finding container d2bfafd0d947e4143f5321e9845abb1c3753b540a3d050c78f32efda3ab5bab8: Status 404 returned error can't find the container with id d2bfafd0d947e4143f5321e9845abb1c3753b540a3d050c78f32efda3ab5bab8
Jul 13 21:17:45.710450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.710369    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2kptt"]
Jul 13 21:17:45.713172 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.713155    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:45.721486 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.721458    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2kptt"]
Jul 13 21:17:45.767844 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.767808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjzpt\" (UniqueName: \"kubernetes.io/projected/1efc2fed-1a71-49a2-a8c2-0839a45a6619-kube-api-access-vjzpt\") pod \"kuadrant-operator-catalog-2kptt\" (UID: \"1efc2fed-1a71-49a2-a8c2-0839a45a6619\") " pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:45.868865 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.868829    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjzpt\" (UniqueName: \"kubernetes.io/projected/1efc2fed-1a71-49a2-a8c2-0839a45a6619-kube-api-access-vjzpt\") pod \"kuadrant-operator-catalog-2kptt\" (UID: \"1efc2fed-1a71-49a2-a8c2-0839a45a6619\") " pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:45.876764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.876740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjzpt\" (UniqueName: \"kubernetes.io/projected/1efc2fed-1a71-49a2-a8c2-0839a45a6619-kube-api-access-vjzpt\") pod \"kuadrant-operator-catalog-2kptt\" (UID: \"1efc2fed-1a71-49a2-a8c2-0839a45a6619\") " pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:45.984312 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:45.984225    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf" event={"ID":"59c1d340-3c32-4c4d-bed1-a6cc90907de7","Type":"ContainerStarted","Data":"d2bfafd0d947e4143f5321e9845abb1c3753b540a3d050c78f32efda3ab5bab8"}
Jul 13 21:17:46.024563 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:46.024533    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:46.163460 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:46.163436    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2kptt"]
Jul 13 21:17:46.165882 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:17:46.165860    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1efc2fed_1a71_49a2_a8c2_0839a45a6619.slice/crio-e21871e21237a6e9317f6b0480b691eb18c6b8fbbe47ed4c8fc3d09f57ed3d4c WatchSource:0}: Error finding container e21871e21237a6e9317f6b0480b691eb18c6b8fbbe47ed4c8fc3d09f57ed3d4c: Status 404 returned error can't find the container with id e21871e21237a6e9317f6b0480b691eb18c6b8fbbe47ed4c8fc3d09f57ed3d4c
Jul 13 21:17:46.992208 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:46.992172    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2kptt" event={"ID":"1efc2fed-1a71-49a2-a8c2-0839a45a6619","Type":"ContainerStarted","Data":"e21871e21237a6e9317f6b0480b691eb18c6b8fbbe47ed4c8fc3d09f57ed3d4c"}
Jul 13 21:17:47.998449 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:47.998412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2kptt" event={"ID":"1efc2fed-1a71-49a2-a8c2-0839a45a6619","Type":"ContainerStarted","Data":"cde1f9a692d0825013b6ef010456f8a167905e4a8057db9119434858cf9927a8"}
Jul 13 21:17:47.999973 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:47.999949    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf" event={"ID":"59c1d340-3c32-4c4d-bed1-a6cc90907de7","Type":"ContainerStarted","Data":"b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757"}
Jul 13 21:17:48.000120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.000036    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf" podUID="59c1d340-3c32-4c4d-bed1-a6cc90907de7" containerName="registry-server" containerID="cri-o://b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757" gracePeriod=2
Jul 13 21:17:48.013873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.013828    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2kptt" podStartSLOduration=1.3922195849999999 podStartE2EDuration="3.013815374s" totalImagesPullingTime="1.621595789s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:17:48.011523967 +0000 UTC m=+375.524273720" watchObservedRunningTime="2026-07-13 21:17:48.013815374 +0000 UTC m=+375.526565114"
Jul 13 21:17:48.026956 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.026908    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf" podStartSLOduration=0.830105958 podStartE2EDuration="3.026895437s" totalImagesPullingTime="2.196789479s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:17:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:17:48.025161271 +0000 UTC m=+375.537911012" watchObservedRunningTime="2026-07-13 21:17:48.026895437 +0000 UTC m=+375.539645186"
Jul 13 21:17:48.240842 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.240817    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:48.292423 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.292395    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q9g5k\" (UniqueName: \"kubernetes.io/projected/59c1d340-3c32-4c4d-bed1-a6cc90907de7-kube-api-access-q9g5k\") pod \"59c1d340-3c32-4c4d-bed1-a6cc90907de7\" (UID: \"59c1d340-3c32-4c4d-bed1-a6cc90907de7\") "
Jul 13 21:17:48.294489 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.294460    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59c1d340-3c32-4c4d-bed1-a6cc90907de7-kube-api-access-q9g5k" (OuterVolumeSpecName: "kube-api-access-q9g5k") pod "59c1d340-3c32-4c4d-bed1-a6cc90907de7" (UID: "59c1d340-3c32-4c4d-bed1-a6cc90907de7"). InnerVolumeSpecName "kube-api-access-q9g5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:17:48.394159 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:48.394085    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q9g5k\" (UniqueName: \"kubernetes.io/projected/59c1d340-3c32-4c4d-bed1-a6cc90907de7-kube-api-access-q9g5k\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:17:49.005653 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.005617    2564 generic.go:356] "Generic (PLEG): container finished" podID="59c1d340-3c32-4c4d-bed1-a6cc90907de7" containerID="b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757" exitCode=0
Jul 13 21:17:49.006088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.005661    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf"
Jul 13 21:17:49.006088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.005705    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf" event={"ID":"59c1d340-3c32-4c4d-bed1-a6cc90907de7","Type":"ContainerDied","Data":"b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757"}
Jul 13 21:17:49.006088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.005752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-zs7nf" event={"ID":"59c1d340-3c32-4c4d-bed1-a6cc90907de7","Type":"ContainerDied","Data":"d2bfafd0d947e4143f5321e9845abb1c3753b540a3d050c78f32efda3ab5bab8"}
Jul 13 21:17:49.006088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.005778    2564 scope.go:117] "RemoveContainer" containerID="b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757"
Jul 13 21:17:49.017238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.017216    2564 scope.go:117] "RemoveContainer" containerID="b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757"
Jul 13 21:17:49.017542 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:17:49.017524    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757\": container with ID starting with b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757 not found: ID does not exist" containerID="b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757"
Jul 13 21:17:49.017587 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.017550    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757"} err="failed to get container status \"b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757\": rpc error: code = NotFound desc = could not find container \"b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757\": container with ID starting with b8edda32c3899218483baee718c76ba756dfec1e5c5f798c937c634a63cb7757 not found: ID does not exist"
Jul 13 21:17:49.028447 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.028409    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zs7nf"]
Jul 13 21:17:49.031657 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.031635    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-zs7nf"]
Jul 13 21:17:49.076988 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:49.076961    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59c1d340-3c32-4c4d-bed1-a6cc90907de7" path="/var/lib/kubelet/pods/59c1d340-3c32-4c4d-bed1-a6cc90907de7/volumes"
Jul 13 21:17:56.025269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:56.025229    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:56.025683 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:56.025303    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:56.049473 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:56.049444    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:17:56.071917 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:17:56.071887    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2kptt"
Jul 13 21:18:15.187167 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.187128    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"]
Jul 13 21:18:15.187614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.187533    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59c1d340-3c32-4c4d-bed1-a6cc90907de7" containerName="registry-server"
Jul 13 21:18:15.187614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.187544    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="59c1d340-3c32-4c4d-bed1-a6cc90907de7" containerName="registry-server"
Jul 13 21:18:15.187614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.187604    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="59c1d340-3c32-4c4d-bed1-a6cc90907de7" containerName="registry-server"
Jul 13 21:18:15.190532 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.190515    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.193335 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.193313    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sghhq\""
Jul 13 21:18:15.206177 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.206155    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"]
Jul 13 21:18:15.243883 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.243852    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cba8c894-a205-49b5-954b-9fff3a25f34b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.244026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.243913    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbzpx\" (UniqueName: \"kubernetes.io/projected/cba8c894-a205-49b5-954b-9fff3a25f34b-kube-api-access-hbzpx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.345303 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.345268    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cba8c894-a205-49b5-954b-9fff3a25f34b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.345479 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.345353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbzpx\" (UniqueName: \"kubernetes.io/projected/cba8c894-a205-49b5-954b-9fff3a25f34b-kube-api-access-hbzpx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.345650 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.345629    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cba8c894-a205-49b5-954b-9fff3a25f34b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.353323 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.353298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbzpx\" (UniqueName: \"kubernetes.io/projected/cba8c894-a205-49b5-954b-9fff3a25f34b-kube-api-access-hbzpx\") pod \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.501185 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.501087    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:15.638591 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:15.638547    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"]
Jul 13 21:18:15.641187 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:15.641161    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcba8c894_a205_49b5_954b_9fff3a25f34b.slice/crio-a2d6b336810a41ee4401c81e04039c02fadb3caf9840bd271fe2fd6db16d592b WatchSource:0}: Error finding container a2d6b336810a41ee4401c81e04039c02fadb3caf9840bd271fe2fd6db16d592b: Status 404 returned error can't find the container with id a2d6b336810a41ee4401c81e04039c02fadb3caf9840bd271fe2fd6db16d592b
Jul 13 21:18:16.143912 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:16.143880    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb" event={"ID":"cba8c894-a205-49b5-954b-9fff3a25f34b","Type":"ContainerStarted","Data":"a2d6b336810a41ee4401c81e04039c02fadb3caf9840bd271fe2fd6db16d592b"}
Jul 13 21:18:17.102932 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.102895    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"]
Jul 13 21:18:17.106622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.106600    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:17.112343 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.112317    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-w85kg\""
Jul 13 21:18:17.125601 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.125566    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"]
Jul 13 21:18:17.164975 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.164941    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btksw\" (UniqueName: \"kubernetes.io/projected/f73323af-e1d9-47f0-ab14-50c5171c9249-kube-api-access-btksw\") pod \"limitador-operator-controller-manager-859d7b78bd-xsj8b\" (UID: \"f73323af-e1d9-47f0-ab14-50c5171c9249\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:17.265675 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.265639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btksw\" (UniqueName: \"kubernetes.io/projected/f73323af-e1d9-47f0-ab14-50c5171c9249-kube-api-access-btksw\") pod \"limitador-operator-controller-manager-859d7b78bd-xsj8b\" (UID: \"f73323af-e1d9-47f0-ab14-50c5171c9249\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:17.274695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.274666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btksw\" (UniqueName: \"kubernetes.io/projected/f73323af-e1d9-47f0-ab14-50c5171c9249-kube-api-access-btksw\") pod \"limitador-operator-controller-manager-859d7b78bd-xsj8b\" (UID: \"f73323af-e1d9-47f0-ab14-50c5171c9249\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:17.420281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.420173    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:17.577881 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:17.577834    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"]
Jul 13 21:18:17.580163 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:17.580122    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf73323af_e1d9_47f0_ab14_50c5171c9249.slice/crio-7f73c6f9c745316be1ee2ae28dad5aac4e3e21605832d43a778feeb6a0c1d20c WatchSource:0}: Error finding container 7f73c6f9c745316be1ee2ae28dad5aac4e3e21605832d43a778feeb6a0c1d20c: Status 404 returned error can't find the container with id 7f73c6f9c745316be1ee2ae28dad5aac4e3e21605832d43a778feeb6a0c1d20c
Jul 13 21:18:18.156770 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:18.156729    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b" event={"ID":"f73323af-e1d9-47f0-ab14-50c5171c9249","Type":"ContainerStarted","Data":"7f73c6f9c745316be1ee2ae28dad5aac4e3e21605832d43a778feeb6a0c1d20c"}
Jul 13 21:18:19.593764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.593722    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"]
Jul 13 21:18:19.597684 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.597658    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:19.600306 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.600283    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-tt6v4\""
Jul 13 21:18:19.609083 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.609039    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"]
Jul 13 21:18:19.692630 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.692525    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgcfq\" (UniqueName: \"kubernetes.io/projected/f1301376-d0ef-45d8-b9a8-a97237d1fbdc-kube-api-access-vgcfq\") pod \"authorino-operator-75cc8d5cb7-7jmsq\" (UID: \"f1301376-d0ef-45d8-b9a8-a97237d1fbdc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:19.793701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.793662    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgcfq\" (UniqueName: \"kubernetes.io/projected/f1301376-d0ef-45d8-b9a8-a97237d1fbdc-kube-api-access-vgcfq\") pod \"authorino-operator-75cc8d5cb7-7jmsq\" (UID: \"f1301376-d0ef-45d8-b9a8-a97237d1fbdc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:19.803648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.803611    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgcfq\" (UniqueName: \"kubernetes.io/projected/f1301376-d0ef-45d8-b9a8-a97237d1fbdc-kube-api-access-vgcfq\") pod \"authorino-operator-75cc8d5cb7-7jmsq\" (UID: \"f1301376-d0ef-45d8-b9a8-a97237d1fbdc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:19.911014 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:19.910974    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:20.070657 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:20.070629    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"]
Jul 13 21:18:20.072532 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:20.072500    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1301376_d0ef_45d8_b9a8_a97237d1fbdc.slice/crio-b36cb9d51327dba96d6ed613612aeac023e04a29610588bc8e530a72a489c1fa WatchSource:0}: Error finding container b36cb9d51327dba96d6ed613612aeac023e04a29610588bc8e530a72a489c1fa: Status 404 returned error can't find the container with id b36cb9d51327dba96d6ed613612aeac023e04a29610588bc8e530a72a489c1fa
Jul 13 21:18:20.175440 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:20.175400    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b" event={"ID":"f73323af-e1d9-47f0-ab14-50c5171c9249","Type":"ContainerStarted","Data":"92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c"}
Jul 13 21:18:20.175626 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:20.175610    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:20.177014 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:20.176977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq" event={"ID":"f1301376-d0ef-45d8-b9a8-a97237d1fbdc","Type":"ContainerStarted","Data":"b36cb9d51327dba96d6ed613612aeac023e04a29610588bc8e530a72a489c1fa"}
Jul 13 21:18:20.192107 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:20.192037    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b" podStartSLOduration=1.360540236 podStartE2EDuration="3.192023419s" totalImagesPullingTime="1.831483183s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:20.190470085 +0000 UTC m=+407.703219828" watchObservedRunningTime="2026-07-13 21:18:20.192023419 +0000 UTC m=+407.704773160"
Jul 13 21:18:22.189738 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:22.189699    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb" event={"ID":"cba8c894-a205-49b5-954b-9fff3a25f34b","Type":"ContainerStarted","Data":"95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c"}
Jul 13 21:18:22.190218 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:22.189777    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:22.212842 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:22.212794    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb" podStartSLOduration=1.422054344 podStartE2EDuration="7.212779393s" totalImagesPullingTime="5.790725049s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:22.210744115 +0000 UTC m=+409.723493857" watchObservedRunningTime="2026-07-13 21:18:22.212779393 +0000 UTC m=+409.725529133"
Jul 13 21:18:23.016137 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.016101    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"]
Jul 13 21:18:23.019837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.019816    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:23.022274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.022250    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-4n55p\""
Jul 13 21:18:23.022274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.022262    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 13 21:18:23.029481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.029460    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"]
Jul 13 21:18:23.133843 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.133807    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjjj9\" (UniqueName: \"kubernetes.io/projected/6b160ad4-2a61-44ba-8360-04d26857359e-kube-api-access-hjjj9\") pod \"dns-operator-controller-manager-864dd78c4c-qj2pl\" (UID: \"6b160ad4-2a61-44ba-8360-04d26857359e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:23.234434 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.234408    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjjj9\" (UniqueName: \"kubernetes.io/projected/6b160ad4-2a61-44ba-8360-04d26857359e-kube-api-access-hjjj9\") pod \"dns-operator-controller-manager-864dd78c4c-qj2pl\" (UID: \"6b160ad4-2a61-44ba-8360-04d26857359e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:23.245450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.245424    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjjj9\" (UniqueName: \"kubernetes.io/projected/6b160ad4-2a61-44ba-8360-04d26857359e-kube-api-access-hjjj9\") pod \"dns-operator-controller-manager-864dd78c4c-qj2pl\" (UID: \"6b160ad4-2a61-44ba-8360-04d26857359e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:23.334486 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.334456    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:23.476691 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:23.476666    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"]
Jul 13 21:18:23.478676 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:23.478648    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b160ad4_2a61_44ba_8360_04d26857359e.slice/crio-8eac9e2fd409bee917f94057dcede961fc09cd4a298cb3396c82f003ff960b65 WatchSource:0}: Error finding container 8eac9e2fd409bee917f94057dcede961fc09cd4a298cb3396c82f003ff960b65: Status 404 returned error can't find the container with id 8eac9e2fd409bee917f94057dcede961fc09cd4a298cb3396c82f003ff960b65
Jul 13 21:18:24.205451 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:24.205412    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl" event={"ID":"6b160ad4-2a61-44ba-8360-04d26857359e","Type":"ContainerStarted","Data":"8eac9e2fd409bee917f94057dcede961fc09cd4a298cb3396c82f003ff960b65"}
Jul 13 21:18:24.207073 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:24.207020    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq" event={"ID":"f1301376-d0ef-45d8-b9a8-a97237d1fbdc","Type":"ContainerStarted","Data":"f1617e660e4257ad09ebf944d036d19a4c6c631c6f5db0fdde915f86cccce8ce"}
Jul 13 21:18:24.207215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:24.207103    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:24.224047 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:24.224001    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq" podStartSLOduration=2.076431918 podStartE2EDuration="5.223986347s" totalImagesPullingTime="3.147554429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:24.222499258 +0000 UTC m=+411.735249000" watchObservedRunningTime="2026-07-13 21:18:24.223986347 +0000 UTC m=+411.736736088"
Jul 13 21:18:26.218882 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:26.218846    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl" event={"ID":"6b160ad4-2a61-44ba-8360-04d26857359e","Type":"ContainerStarted","Data":"a6b9079307574c98938bffacddfa87c6b36db24cfee05c64628b9841774692d0"}
Jul 13 21:18:26.219306 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:26.218998    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:26.236518 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:26.236479    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl" podStartSLOduration=2.215409354 podStartE2EDuration="4.236464807s" totalImagesPullingTime="2.021055453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:26.234322016 +0000 UTC m=+413.747071758" watchObservedRunningTime="2026-07-13 21:18:26.236464807 +0000 UTC m=+413.749214547"
Jul 13 21:18:31.184903 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:31.184873    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:33.196716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:33.196682    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:34.573339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.573310    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"]
Jul 13 21:18:34.578365 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.578345    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.589981 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.589955    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"]
Jul 13 21:18:34.735960 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.735926    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.736148 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.736014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh8zd\" (UniqueName: \"kubernetes.io/projected/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-kube-api-access-lh8zd\") pod \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.837273 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.837188    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.837413 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.837312    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh8zd\" (UniqueName: \"kubernetes.io/projected/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-kube-api-access-lh8zd\") pod \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.837608 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.837592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.846328 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.846297    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh8zd\" (UniqueName: \"kubernetes.io/projected/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-kube-api-access-lh8zd\") pod \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.889801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.889769    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:34.964547 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.964514    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"]
Jul 13 21:18:34.977022 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.976993    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"]
Jul 13 21:18:34.986720 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.986693    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"]
Jul 13 21:18:34.986954 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.986909    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" containerName="manager" containerID="cri-o://95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c" gracePeriod=2
Jul 13 21:18:34.989883 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:34.989859    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"]
Jul 13 21:18:35.000188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.000167    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"]
Jul 13 21:18:35.000345 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.000293    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.008154 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.008113    2564 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"cba8c894-a205-49b5-954b-9fff3a25f34b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\": pods \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:35.012220 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.012188    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:35.013585 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.013559    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"]
Jul 13 21:18:35.016170 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.016137    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"]
Jul 13 21:18:35.016596 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.016579    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" containerName="manager"
Jul 13 21:18:35.016663 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.016600    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" containerName="manager"
Jul 13 21:18:35.016738 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.016726    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" containerName="manager"
Jul 13 21:18:35.019763 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.019747    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"]
Jul 13 21:18:35.019899 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.019875    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.019980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.019947    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" containerName="manager" containerID="cri-o://92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c" gracePeriod=2
Jul 13 21:18:35.031458 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.031433    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"]
Jul 13 21:18:35.033685 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.033667    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"]
Jul 13 21:18:35.036484 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.036275    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:35.040458 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.040435    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"]
Jul 13 21:18:35.040821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.040808    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" containerName="manager"
Jul 13 21:18:35.040821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.040821    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" containerName="manager"
Jul 13 21:18:35.040931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.040893    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" containerName="manager"
Jul 13 21:18:35.044907 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.044889    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:35.055913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.055655    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"]
Jul 13 21:18:35.060388 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.060353    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xsj8b\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:35.062322 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.062290    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xsj8b\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:35.064371 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.064334    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-sbbtb\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:35.140574 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.140508    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wwblm\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.140695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.140669    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bfee83-3195-4588-80e7-4821ed6606a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m9sr6\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.140759 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.140720    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d7ld\" (UniqueName: \"kubernetes.io/projected/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-kube-api-access-4d7ld\") pod \"kuadrant-operator-controller-manager-55467d9f7-wwblm\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.140894 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.140864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9bvx\" (UniqueName: \"kubernetes.io/projected/93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8-kube-api-access-s9bvx\") pod \"limitador-operator-controller-manager-859d7b78bd-2xrrf\" (UID: \"93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:35.140945 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.140931    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8smc\" (UniqueName: \"kubernetes.io/projected/76bfee83-3195-4588-80e7-4821ed6606a3-kube-api-access-t8smc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m9sr6\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.192221 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.192171    2564 log.go:32] "RunPodSandbox from runtime service failed" err=<
Jul 13 21:18:35.192221 ip-10-0-131-147 kubenswrapper[2564]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-pf6pj_kuadrant-system_d570b22f-1040-4cde-bd8f-5cfdb0dbeae5_0(eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-pf6pj to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c" Netns:"/var/run/netns/0a79822d-76ba-4df8-b3cd-ad1dec7ff337" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-pf6pj;K8S_POD_INFRA_CONTAINER_ID=eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c;K8S_POD_UID=d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:28 [10.134.0.40/23]
Jul 13 21:18:35.192221 ip-10-0-131-147 kubenswrapper[2564]:         '
Jul 13 21:18:35.192221 ip-10-0-131-147 kubenswrapper[2564]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 13 21:18:35.192221 ip-10-0-131-147 kubenswrapper[2564]:  >
Jul 13 21:18:35.192561 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.192245    2564 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Jul 13 21:18:35.192561 ip-10-0-131-147 kubenswrapper[2564]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-pf6pj_kuadrant-system_d570b22f-1040-4cde-bd8f-5cfdb0dbeae5_0(eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-pf6pj to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c" Netns:"/var/run/netns/0a79822d-76ba-4df8-b3cd-ad1dec7ff337" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-55467d9f7-pf6pj;K8S_POD_INFRA_CONTAINER_ID=eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c;K8S_POD_UID=d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj] networking: [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c network default NAD default NAD key default] [kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj eb650a79ed0163a9576ab40b83948c6e0623a661d64a7316971db78037a1355c network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:28 [10.134.0.40/23]
Jul 13 21:18:35.192561 ip-10-0-131-147 kubenswrapper[2564]:         '
Jul 13 21:18:35.192561 ip-10-0-131-147 kubenswrapper[2564]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Jul 13 21:18:35.192561 ip-10-0-131-147 kubenswrapper[2564]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:35.215167 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.215140    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-7jmsq"
Jul 13 21:18:35.242261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242230    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bfee83-3195-4588-80e7-4821ed6606a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m9sr6\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.242376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242293    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d7ld\" (UniqueName: \"kubernetes.io/projected/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-kube-api-access-4d7ld\") pod \"kuadrant-operator-controller-manager-55467d9f7-wwblm\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.242376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242353    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9bvx\" (UniqueName: \"kubernetes.io/projected/93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8-kube-api-access-s9bvx\") pod \"limitador-operator-controller-manager-859d7b78bd-2xrrf\" (UID: \"93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:35.242495 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242390    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8smc\" (UniqueName: \"kubernetes.io/projected/76bfee83-3195-4588-80e7-4821ed6606a3-kube-api-access-t8smc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m9sr6\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.242495 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242451    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wwblm\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.242601 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242582    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bfee83-3195-4588-80e7-4821ed6606a3-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m9sr6\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.242926 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.242907    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-wwblm\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.250960 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.250937    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:35.252226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.252200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8smc\" (UniqueName: \"kubernetes.io/projected/76bfee83-3195-4588-80e7-4821ed6606a3-kube-api-access-t8smc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m9sr6\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.252390 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.252346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9bvx\" (UniqueName: \"kubernetes.io/projected/93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8-kube-api-access-s9bvx\") pod \"limitador-operator-controller-manager-859d7b78bd-2xrrf\" (UID: \"93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:35.252512 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.252489    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d7ld\" (UniqueName: \"kubernetes.io/projected/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-kube-api-access-4d7ld\") pod \"kuadrant-operator-controller-manager-55467d9f7-wwblm\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.257790 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.257768    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:35.272477 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.272449    2564 generic.go:356] "Generic (PLEG): container finished" podID="f73323af-e1d9-47f0-ab14-50c5171c9249" containerID="92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c" exitCode=0
Jul 13 21:18:35.272583 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.272486    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xsj8b"
Jul 13 21:18:35.272583 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.272557    2564 scope.go:117] "RemoveContainer" containerID="92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c"
Jul 13 21:18:35.273941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.273910    2564 generic.go:356] "Generic (PLEG): container finished" podID="cba8c894-a205-49b5-954b-9fff3a25f34b" containerID="95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c" exitCode=0
Jul 13 21:18:35.274030 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.273981    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-sbbtb"
Jul 13 21:18:35.274171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.274158    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:35.282049 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.282037    2564 scope.go:117] "RemoveContainer" containerID="92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c"
Jul 13 21:18:35.282333 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.282316    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c\": container with ID starting with 92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c not found: ID does not exist" containerID="92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c"
Jul 13 21:18:35.282377 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.282342    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c"} err="failed to get container status \"92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c\": rpc error: code = NotFound desc = could not find container \"92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c\": container with ID starting with 92c0e41bd8d6498ae6c27106070c810a6640a003072b1af3d495ebd782d9599c not found: ID does not exist"
Jul 13 21:18:35.282377 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.282358    2564 scope.go:117] "RemoveContainer" containerID="95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c"
Jul 13 21:18:35.282674 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.282659    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:35.284952 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.284928    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:35.290190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.290175    2564 scope.go:117] "RemoveContainer" containerID="95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c"
Jul 13 21:18:35.290429 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.290409    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c\": container with ID starting with 95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c not found: ID does not exist" containerID="95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c"
Jul 13 21:18:35.290486 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.290435    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c"} err="failed to get container status \"95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c\": rpc error: code = NotFound desc = could not find container \"95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c\": container with ID starting with 95c1d2968b0e963e8ec8b7c70d14ae3798a52b8f9b928b348a9d7132b129451c not found: ID does not exist"
Jul 13 21:18:35.343800 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.343775    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-btksw\" (UniqueName: \"kubernetes.io/projected/f73323af-e1d9-47f0-ab14-50c5171c9249-kube-api-access-btksw\") pod \"f73323af-e1d9-47f0-ab14-50c5171c9249\" (UID: \"f73323af-e1d9-47f0-ab14-50c5171c9249\") "
Jul 13 21:18:35.343892 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.343852    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbzpx\" (UniqueName: \"kubernetes.io/projected/cba8c894-a205-49b5-954b-9fff3a25f34b-kube-api-access-hbzpx\") pod \"cba8c894-a205-49b5-954b-9fff3a25f34b\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") "
Jul 13 21:18:35.343939 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.343929    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cba8c894-a205-49b5-954b-9fff3a25f34b-extensions-socket-volume\") pod \"cba8c894-a205-49b5-954b-9fff3a25f34b\" (UID: \"cba8c894-a205-49b5-954b-9fff3a25f34b\") "
Jul 13 21:18:35.344315 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.344291    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cba8c894-a205-49b5-954b-9fff3a25f34b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "cba8c894-a205-49b5-954b-9fff3a25f34b" (UID: "cba8c894-a205-49b5-954b-9fff3a25f34b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:18:35.345836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.345816    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f73323af-e1d9-47f0-ab14-50c5171c9249-kube-api-access-btksw" (OuterVolumeSpecName: "kube-api-access-btksw") pod "f73323af-e1d9-47f0-ab14-50c5171c9249" (UID: "f73323af-e1d9-47f0-ab14-50c5171c9249"). InnerVolumeSpecName "kube-api-access-btksw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:18:35.345836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.345825    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cba8c894-a205-49b5-954b-9fff3a25f34b-kube-api-access-hbzpx" (OuterVolumeSpecName: "kube-api-access-hbzpx") pod "cba8c894-a205-49b5-954b-9fff3a25f34b" (UID: "cba8c894-a205-49b5-954b-9fff3a25f34b"). InnerVolumeSpecName "kube-api-access-hbzpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:18:35.425040 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.424964    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:35.433490 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.433470    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:35.442419 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.442395    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:35.444693 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.444672    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-extensions-socket-volume\") pod \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") "
Jul 13 21:18:35.444804 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.444793    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lh8zd\" (UniqueName: \"kubernetes.io/projected/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-kube-api-access-lh8zd\") pod \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\" (UID: \"d570b22f-1040-4cde-bd8f-5cfdb0dbeae5\") "
Jul 13 21:18:35.444986 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.444971    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/cba8c894-a205-49b5-954b-9fff3a25f34b-extensions-socket-volume\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:35.445047 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.444988    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-btksw\" (UniqueName: \"kubernetes.io/projected/f73323af-e1d9-47f0-ab14-50c5171c9249-kube-api-access-btksw\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:35.445047 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.444997    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbzpx\" (UniqueName: \"kubernetes.io/projected/cba8c894-a205-49b5-954b-9fff3a25f34b-kube-api-access-hbzpx\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:35.445176 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.445038    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" (UID: "d570b22f-1040-4cde-bd8f-5cfdb0dbeae5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:18:35.447017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.446990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-kube-api-access-lh8zd" (OuterVolumeSpecName: "kube-api-access-lh8zd") pod "d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" (UID: "d570b22f-1040-4cde-bd8f-5cfdb0dbeae5"). InnerVolumeSpecName "kube-api-access-lh8zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:18:35.545950 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.545921    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-extensions-socket-volume\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:35.546084 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.545956    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lh8zd\" (UniqueName: \"kubernetes.io/projected/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5-kube-api-access-lh8zd\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:35.588840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.588744    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"]
Jul 13 21:18:35.589497 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.589324    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:35.590664 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:35.590640    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573ef60e_f4bd_4b20_bc7b_f8b9b3902502.slice/crio-b78d4b77c344ff48c5ee921ac87591f06d0189822291e70dcf72e55dceda004f WatchSource:0}: Error finding container b78d4b77c344ff48c5ee921ac87591f06d0189822291e70dcf72e55dceda004f: Status 404 returned error can't find the container with id b78d4b77c344ff48c5ee921ac87591f06d0189822291e70dcf72e55dceda004f
Jul 13 21:18:35.591393 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:35.591360    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:35.818856 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.818832    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"]
Jul 13 21:18:35.820714 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:35.820686    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f9dd3b_e5fe_4f0d_b326_2ce93bc060e8.slice/crio-38b48d2bcaee59f9e5a79c2fa6459abf64f845b08c0d09cbe18a6bd431efd8a3 WatchSource:0}: Error finding container 38b48d2bcaee59f9e5a79c2fa6459abf64f845b08c0d09cbe18a6bd431efd8a3: Status 404 returned error can't find the container with id 38b48d2bcaee59f9e5a79c2fa6459abf64f845b08c0d09cbe18a6bd431efd8a3
Jul 13 21:18:35.825529 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:35.825505    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"]
Jul 13 21:18:35.826307 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:35.826278    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76bfee83_3195_4588_80e7_4821ed6606a3.slice/crio-80f6b1f6fa22113827b9391ebc677c1b5428ec77b200a94719d46535635828a4 WatchSource:0}: Error finding container 80f6b1f6fa22113827b9391ebc677c1b5428ec77b200a94719d46535635828a4: Status 404 returned error can't find the container with id 80f6b1f6fa22113827b9391ebc677c1b5428ec77b200a94719d46535635828a4
Jul 13 21:18:36.282378 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.282341    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf" event={"ID":"93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8","Type":"ContainerStarted","Data":"7147c95e1e39279829ddb8f60661e2254911c90c18cb0ca2bcb015ff92886b95"}
Jul 13 21:18:36.282548 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.282425    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf" event={"ID":"93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8","Type":"ContainerStarted","Data":"38b48d2bcaee59f9e5a79c2fa6459abf64f845b08c0d09cbe18a6bd431efd8a3"}
Jul 13 21:18:36.282548 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.282450    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:36.283911 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.283889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm" event={"ID":"573ef60e-f4bd-4b20-bc7b-f8b9b3902502","Type":"ContainerStarted","Data":"e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc"}
Jul 13 21:18:36.284034 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.283933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm" event={"ID":"573ef60e-f4bd-4b20-bc7b-f8b9b3902502","Type":"ContainerStarted","Data":"b78d4b77c344ff48c5ee921ac87591f06d0189822291e70dcf72e55dceda004f"}
Jul 13 21:18:36.284034 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.283960    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:36.285045 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:36.285020    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:36.285438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.285423    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:36.285487 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.285449    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6" event={"ID":"76bfee83-3195-4588-80e7-4821ed6606a3","Type":"ContainerStarted","Data":"182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51"}
Jul 13 21:18:36.285526 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.285486    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6" event={"ID":"76bfee83-3195-4588-80e7-4821ed6606a3","Type":"ContainerStarted","Data":"80f6b1f6fa22113827b9391ebc677c1b5428ec77b200a94719d46535635828a4"}
Jul 13 21:18:36.285568 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.285528    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:36.305494 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.305450    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf" podStartSLOduration=1.305435282 podStartE2EDuration="1.305435282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:36.303275772 +0000 UTC m=+423.816025517" watchObservedRunningTime="2026-07-13 21:18:36.305435282 +0000 UTC m=+423.818185022"
Jul 13 21:18:36.324433 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.324387    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm" podStartSLOduration=2.324371187 podStartE2EDuration="2.324371187s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:36.322419133 +0000 UTC m=+423.835168874" watchObservedRunningTime="2026-07-13 21:18:36.324371187 +0000 UTC m=+423.837120928"
Jul 13 21:18:36.341554 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:36.341507    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6" podStartSLOduration=2.341491695 podStartE2EDuration="2.341491695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:36.340407077 +0000 UTC m=+423.853156817" watchObservedRunningTime="2026-07-13 21:18:36.341491695 +0000 UTC m=+423.854241436"
Jul 13 21:18:36.342649 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:36.342623    2564 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-pf6pj\" is forbidden: User \"system:node:ip-10-0-131-147.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-147.ec2.internal' and this object" podUID="d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-pf6pj"
Jul 13 21:18:37.077146 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:37.077113    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cba8c894-a205-49b5-954b-9fff3a25f34b" path="/var/lib/kubelet/pods/cba8c894-a205-49b5-954b-9fff3a25f34b/volumes"
Jul 13 21:18:37.077503 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:37.077458    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d570b22f-1040-4cde-bd8f-5cfdb0dbeae5" path="/var/lib/kubelet/pods/d570b22f-1040-4cde-bd8f-5cfdb0dbeae5/volumes"
Jul 13 21:18:37.077654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:37.077640    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f73323af-e1d9-47f0-ab14-50c5171c9249" path="/var/lib/kubelet/pods/f73323af-e1d9-47f0-ab14-50c5171c9249/volumes"
Jul 13 21:18:37.226000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:37.225972    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-qj2pl"
Jul 13 21:18:47.294030 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.293999    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-2xrrf"
Jul 13 21:18:47.294486 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.294203    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:47.294486 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.294276    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:47.373152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.373124    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"]
Jul 13 21:18:47.373359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.373320    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6" podUID="76bfee83-3195-4588-80e7-4821ed6606a3" containerName="manager" containerID="cri-o://182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51" gracePeriod=10
Jul 13 21:18:47.632174 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.632150    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:47.685769 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.685729    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"]
Jul 13 21:18:47.686186 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.686168    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76bfee83-3195-4588-80e7-4821ed6606a3" containerName="manager"
Jul 13 21:18:47.686186 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.686184    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="76bfee83-3195-4588-80e7-4821ed6606a3" containerName="manager"
Jul 13 21:18:47.686349 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.686241    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="76bfee83-3195-4588-80e7-4821ed6606a3" containerName="manager"
Jul 13 21:18:47.689189 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.689169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.709715 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.709659    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"]
Jul 13 21:18:47.752732 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.752702    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t8smc\" (UniqueName: \"kubernetes.io/projected/76bfee83-3195-4588-80e7-4821ed6606a3-kube-api-access-t8smc\") pod \"76bfee83-3195-4588-80e7-4821ed6606a3\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") "
Jul 13 21:18:47.752860 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.752801    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bfee83-3195-4588-80e7-4821ed6606a3-extensions-socket-volume\") pod \"76bfee83-3195-4588-80e7-4821ed6606a3\" (UID: \"76bfee83-3195-4588-80e7-4821ed6606a3\") "
Jul 13 21:18:47.753170 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.753149    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76bfee83-3195-4588-80e7-4821ed6606a3-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "76bfee83-3195-4588-80e7-4821ed6606a3" (UID: "76bfee83-3195-4588-80e7-4821ed6606a3"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:18:47.754618 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.754599    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76bfee83-3195-4588-80e7-4821ed6606a3-kube-api-access-t8smc" (OuterVolumeSpecName: "kube-api-access-t8smc") pod "76bfee83-3195-4588-80e7-4821ed6606a3" (UID: "76bfee83-3195-4588-80e7-4821ed6606a3"). InnerVolumeSpecName "kube-api-access-t8smc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:18:47.854443 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.854415    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e16b212e-63e7-431f-9fde-e44fbd5dda47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ljztn\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.854588 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.854541    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tl64b\" (UniqueName: \"kubernetes.io/projected/e16b212e-63e7-431f-9fde-e44fbd5dda47-kube-api-access-tl64b\") pod \"kuadrant-operator-controller-manager-86878c769f-ljztn\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.854647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.854612    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t8smc\" (UniqueName: \"kubernetes.io/projected/76bfee83-3195-4588-80e7-4821ed6606a3-kube-api-access-t8smc\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:47.854647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.854622    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/76bfee83-3195-4588-80e7-4821ed6606a3-extensions-socket-volume\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:47.955955 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.955916    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e16b212e-63e7-431f-9fde-e44fbd5dda47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ljztn\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.956212 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.955994    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tl64b\" (UniqueName: \"kubernetes.io/projected/e16b212e-63e7-431f-9fde-e44fbd5dda47-kube-api-access-tl64b\") pod \"kuadrant-operator-controller-manager-86878c769f-ljztn\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.956406 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.956382    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e16b212e-63e7-431f-9fde-e44fbd5dda47-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-ljztn\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.968042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.968015    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tl64b\" (UniqueName: \"kubernetes.io/projected/e16b212e-63e7-431f-9fde-e44fbd5dda47-kube-api-access-tl64b\") pod \"kuadrant-operator-controller-manager-86878c769f-ljztn\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:47.999867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:47.999838    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:48.137077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.137040    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"]
Jul 13 21:18:48.140030 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:18:48.140001    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16b212e_63e7_431f_9fde_e44fbd5dda47.slice/crio-2db596e881f67e66e28a2415707e2135d2820c0dff2aed76fb4882d0786f2e08 WatchSource:0}: Error finding container 2db596e881f67e66e28a2415707e2135d2820c0dff2aed76fb4882d0786f2e08: Status 404 returned error can't find the container with id 2db596e881f67e66e28a2415707e2135d2820c0dff2aed76fb4882d0786f2e08
Jul 13 21:18:48.353723 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.353679    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn" event={"ID":"e16b212e-63e7-431f-9fde-e44fbd5dda47","Type":"ContainerStarted","Data":"41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04"}
Jul 13 21:18:48.354197 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.353752    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn" event={"ID":"e16b212e-63e7-431f-9fde-e44fbd5dda47","Type":"ContainerStarted","Data":"2db596e881f67e66e28a2415707e2135d2820c0dff2aed76fb4882d0786f2e08"}
Jul 13 21:18:48.354197 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.353869    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:48.355022 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.355001    2564 generic.go:356] "Generic (PLEG): container finished" podID="76bfee83-3195-4588-80e7-4821ed6606a3" containerID="182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51" exitCode=0
Jul 13 21:18:48.355133 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.355039    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"
Jul 13 21:18:48.355133 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.355075    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6" event={"ID":"76bfee83-3195-4588-80e7-4821ed6606a3","Type":"ContainerDied","Data":"182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51"}
Jul 13 21:18:48.355133 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.355129    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6" event={"ID":"76bfee83-3195-4588-80e7-4821ed6606a3","Type":"ContainerDied","Data":"80f6b1f6fa22113827b9391ebc677c1b5428ec77b200a94719d46535635828a4"}
Jul 13 21:18:48.355232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.355155    2564 scope.go:117] "RemoveContainer" containerID="182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51"
Jul 13 21:18:48.363868 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.363852    2564 scope.go:117] "RemoveContainer" containerID="182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51"
Jul 13 21:18:48.364175 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:18:48.364147    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51\": container with ID starting with 182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51 not found: ID does not exist" containerID="182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51"
Jul 13 21:18:48.364245 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.364182    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51"} err="failed to get container status \"182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51\": rpc error: code = NotFound desc = could not find container \"182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51\": container with ID starting with 182b68e72007d75b5a0233cafdb902964f1b91883e533dc2c4fff51e44c99d51 not found: ID does not exist"
Jul 13 21:18:48.372879 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.372837    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn" podStartSLOduration=1.372824012 podStartE2EDuration="1.372824012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:18:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:18:48.371856737 +0000 UTC m=+435.884606478" watchObservedRunningTime="2026-07-13 21:18:48.372824012 +0000 UTC m=+435.885573753"
Jul 13 21:18:48.388023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.387973    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"]
Jul 13 21:18:48.393126 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:48.393107    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m9sr6"]
Jul 13 21:18:49.076841 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:49.076808    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76bfee83-3195-4588-80e7-4821ed6606a3" path="/var/lib/kubelet/pods/76bfee83-3195-4588-80e7-4821ed6606a3/volumes"
Jul 13 21:18:59.364279 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.364208    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:18:59.413163 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.413131    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"]
Jul 13 21:18:59.413388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.413357    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm" podUID="573ef60e-f4bd-4b20-bc7b-f8b9b3902502" containerName="manager" containerID="cri-o://e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc" gracePeriod=10
Jul 13 21:18:59.672866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.672840    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:18:59.754600 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.754572    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d7ld\" (UniqueName: \"kubernetes.io/projected/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-kube-api-access-4d7ld\") pod \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") "
Jul 13 21:18:59.754739 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.754660    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-extensions-socket-volume\") pod \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\" (UID: \"573ef60e-f4bd-4b20-bc7b-f8b9b3902502\") "
Jul 13 21:18:59.755050 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.755018    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "573ef60e-f4bd-4b20-bc7b-f8b9b3902502" (UID: "573ef60e-f4bd-4b20-bc7b-f8b9b3902502"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:18:59.756622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.756599    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-kube-api-access-4d7ld" (OuterVolumeSpecName: "kube-api-access-4d7ld") pod "573ef60e-f4bd-4b20-bc7b-f8b9b3902502" (UID: "573ef60e-f4bd-4b20-bc7b-f8b9b3902502"). InnerVolumeSpecName "kube-api-access-4d7ld". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:18:59.855412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.855372    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-extensions-socket-volume\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:18:59.855412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:18:59.855407    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d7ld\" (UniqueName: \"kubernetes.io/projected/573ef60e-f4bd-4b20-bc7b-f8b9b3902502-kube-api-access-4d7ld\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:00.419909 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.419868    2564 generic.go:356] "Generic (PLEG): container finished" podID="573ef60e-f4bd-4b20-bc7b-f8b9b3902502" containerID="e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc" exitCode=0
Jul 13 21:19:00.420352 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.419928    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"
Jul 13 21:19:00.420352 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.419958    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm" event={"ID":"573ef60e-f4bd-4b20-bc7b-f8b9b3902502","Type":"ContainerDied","Data":"e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc"}
Jul 13 21:19:00.420352 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.420029    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm" event={"ID":"573ef60e-f4bd-4b20-bc7b-f8b9b3902502","Type":"ContainerDied","Data":"b78d4b77c344ff48c5ee921ac87591f06d0189822291e70dcf72e55dceda004f"}
Jul 13 21:19:00.420352 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.420078    2564 scope.go:117] "RemoveContainer" containerID="e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc"
Jul 13 21:19:00.429148 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.429130    2564 scope.go:117] "RemoveContainer" containerID="e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc"
Jul 13 21:19:00.429383 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:19:00.429367    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc\": container with ID starting with e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc not found: ID does not exist" containerID="e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc"
Jul 13 21:19:00.429447 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.429389    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc"} err="failed to get container status \"e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc\": rpc error: code = NotFound desc = could not find container \"e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc\": container with ID starting with e56dd7239a94c83435da82a473c553be45651722f38baa689a81c9cadb158bcc not found: ID does not exist"
Jul 13 21:19:00.441873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.441854    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"]
Jul 13 21:19:00.449648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:00.449620    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-wwblm"]
Jul 13 21:19:01.076984 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:01.076952    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="573ef60e-f4bd-4b20-bc7b-f8b9b3902502" path="/var/lib/kubelet/pods/573ef60e-f4bd-4b20-bc7b-f8b9b3902502/volumes"
Jul 13 21:19:03.735883 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.735853    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"]
Jul 13 21:19:03.736537 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.736518    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573ef60e-f4bd-4b20-bc7b-f8b9b3902502" containerName="manager"
Jul 13 21:19:03.736588 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.736539    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="573ef60e-f4bd-4b20-bc7b-f8b9b3902502" containerName="manager"
Jul 13 21:19:03.736622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.736612    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="573ef60e-f4bd-4b20-bc7b-f8b9b3902502" containerName="manager"
Jul 13 21:19:03.742282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.742266    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.744585 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.744568    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-2rcjc\""
Jul 13 21:19:03.751613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.751592    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"]
Jul 13 21:19:03.790429 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790392    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790441    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790529    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790585    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790628    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790645    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4348c7b-31e4-41e9-abc8-f780cc694d57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790761 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790664    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790761 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790684    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.790826 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.790788    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-874rz\" (UniqueName: \"kubernetes.io/projected/e4348c7b-31e4-41e9-abc8-f780cc694d57-kube-api-access-874rz\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.891773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891743    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.891915 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.891915 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4348c7b-31e4-41e9-abc8-f780cc694d57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.891915 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891812    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.891915 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891857    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891937    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-874rz\" (UniqueName: \"kubernetes.io/projected/e4348c7b-31e4-41e9-abc8-f780cc694d57-kube-api-access-874rz\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.891981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892513 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.892175    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.892543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.892804    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.892827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.892904    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.892983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.892941    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.893338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.893152    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4348c7b-31e4-41e9-abc8-f780cc694d57-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.895527 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.895499    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.900399 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.900378    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.901490 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.901463    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-874rz\" (UniqueName: \"kubernetes.io/projected/e4348c7b-31e4-41e9-abc8-f780cc694d57-kube-api-access-874rz\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:03.902309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:03.902284    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4348c7b-31e4-41e9-abc8-f780cc694d57-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vgvkp\" (UID: \"e4348c7b-31e4-41e9-abc8-f780cc694d57\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:04.053182 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.053104    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:04.192541 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.192514    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"]
Jul 13 21:19:04.194481 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:19:04.194454    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4348c7b_31e4_41e9_abc8_f780cc694d57.slice/crio-adeb59b4695063ec58bfd3de52be679e1346873f62ec09dd501cd2fb01a5c3c0 WatchSource:0}: Error finding container adeb59b4695063ec58bfd3de52be679e1346873f62ec09dd501cd2fb01a5c3c0: Status 404 returned error can't find the container with id adeb59b4695063ec58bfd3de52be679e1346873f62ec09dd501cd2fb01a5c3c0
Jul 13 21:19:04.196772 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.196734    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:19:04.196835 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.196821    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:19:04.196886 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.196861    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:19:04.446292 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.446259    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" event={"ID":"e4348c7b-31e4-41e9-abc8-f780cc694d57","Type":"ContainerStarted","Data":"0f1bfcc962b88b1f100c91bfba862277660230b23551417e17c897792816100f"}
Jul 13 21:19:04.446497 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.446351    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" event={"ID":"e4348c7b-31e4-41e9-abc8-f780cc694d57","Type":"ContainerStarted","Data":"adeb59b4695063ec58bfd3de52be679e1346873f62ec09dd501cd2fb01a5c3c0"}
Jul 13 21:19:04.467772 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:04.467710    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" podStartSLOduration=1.467688916 podStartE2EDuration="1.467688916s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:04.465018829 +0000 UTC m=+451.977768572" watchObservedRunningTime="2026-07-13 21:19:04.467688916 +0000 UTC m=+451.980438746"
Jul 13 21:19:05.054114 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:05.054078    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:05.058704 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:05.058680    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:05.451421 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:05.451386    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:05.452183 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:05.452165    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:19:08.215689 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.215658    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-69969d6f4c-sjv96"]
Jul 13 21:19:08.220304 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.220280    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.231330 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.231308    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69969d6f4c-sjv96"]
Jul 13 21:19:08.331592 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331571    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-oauth-config\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.331592 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-serving-cert\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.331745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331622    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-service-ca\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.331745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331690    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-config\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.331745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331719    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxgk\" (UniqueName: \"kubernetes.io/projected/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-kube-api-access-gcxgk\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.331846 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331748    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-oauth-serving-cert\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.331846 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.331778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-trusted-ca-bundle\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.432829 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.432804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-oauth-config\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.432920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.432837    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-serving-cert\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.432920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.432853    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-service-ca\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.432920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.432871    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-config\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.432920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.432888    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxgk\" (UniqueName: \"kubernetes.io/projected/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-kube-api-access-gcxgk\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.433115 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.433072    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-oauth-serving-cert\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.433174 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.433143    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-trusted-ca-bundle\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.433622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.433600    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-config\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.433706 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.433637    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-service-ca\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.433757 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.433740    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-oauth-serving-cert\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.434169 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.434149    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-trusted-ca-bundle\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.435281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.435260    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-oauth-config\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.435398 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.435383    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-console-serving-cert\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.439625 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.439607    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxgk\" (UniqueName: \"kubernetes.io/projected/5a477817-4523-4ca1-a8bc-a53d3cb05ab6-kube-api-access-gcxgk\") pod \"console-69969d6f4c-sjv96\" (UID: \"5a477817-4523-4ca1-a8bc-a53d3cb05ab6\") " pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.531222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.531152    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:08.668367 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:08.668340    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-69969d6f4c-sjv96"]
Jul 13 21:19:08.670095 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:19:08.670035    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a477817_4523_4ca1_a8bc_a53d3cb05ab6.slice/crio-b20557727af88434e9cd06ee58f2dafab9483d6074c1e1b77bfffa4871026091 WatchSource:0}: Error finding container b20557727af88434e9cd06ee58f2dafab9483d6074c1e1b77bfffa4871026091: Status 404 returned error can't find the container with id b20557727af88434e9cd06ee58f2dafab9483d6074c1e1b77bfffa4871026091
Jul 13 21:19:09.475919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:09.475889    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69969d6f4c-sjv96" event={"ID":"5a477817-4523-4ca1-a8bc-a53d3cb05ab6","Type":"ContainerStarted","Data":"9e116233f767727536c644273971ac576fd2ecadc7280cbb41ae29d79a6bac65"}
Jul 13 21:19:09.476296 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:09.475933    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-69969d6f4c-sjv96" event={"ID":"5a477817-4523-4ca1-a8bc-a53d3cb05ab6","Type":"ContainerStarted","Data":"b20557727af88434e9cd06ee58f2dafab9483d6074c1e1b77bfffa4871026091"}
Jul 13 21:19:09.494676 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:09.494629    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-69969d6f4c-sjv96" podStartSLOduration=1.494613978 podStartE2EDuration="1.494613978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:09.492721699 +0000 UTC m=+457.005471452" watchObservedRunningTime="2026-07-13 21:19:09.494613978 +0000 UTC m=+457.007363719"
Jul 13 21:19:18.531563 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.531529    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:18.532230 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.531834    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:18.537422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.537398    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:18.769456 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.769423    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:19:18.773984 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.773961    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:18.776488 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.776465    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 13 21:19:18.776598 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.776561    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v2h48\""
Jul 13 21:19:18.779671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.779652    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:19:18.805173 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.805150    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:19:18.828721 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.828697    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a8ad431-364b-4ffe-a252-a5dffede995e-config-file\") pod \"limitador-limitador-54d6d988d5-t7ns6\" (UID: \"8a8ad431-364b-4ffe-a252-a5dffede995e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:18.828874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.828854    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v74v2\" (UniqueName: \"kubernetes.io/projected/8a8ad431-364b-4ffe-a252-a5dffede995e-kube-api-access-v74v2\") pod \"limitador-limitador-54d6d988d5-t7ns6\" (UID: \"8a8ad431-364b-4ffe-a252-a5dffede995e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:18.929590 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.929558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v74v2\" (UniqueName: \"kubernetes.io/projected/8a8ad431-364b-4ffe-a252-a5dffede995e-kube-api-access-v74v2\") pod \"limitador-limitador-54d6d988d5-t7ns6\" (UID: \"8a8ad431-364b-4ffe-a252-a5dffede995e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:18.929746 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.929611    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a8ad431-364b-4ffe-a252-a5dffede995e-config-file\") pod \"limitador-limitador-54d6d988d5-t7ns6\" (UID: \"8a8ad431-364b-4ffe-a252-a5dffede995e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:18.930255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.930235    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/8a8ad431-364b-4ffe-a252-a5dffede995e-config-file\") pod \"limitador-limitador-54d6d988d5-t7ns6\" (UID: \"8a8ad431-364b-4ffe-a252-a5dffede995e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:18.938429 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:18.938406    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v74v2\" (UniqueName: \"kubernetes.io/projected/8a8ad431-364b-4ffe-a252-a5dffede995e-kube-api-access-v74v2\") pod \"limitador-limitador-54d6d988d5-t7ns6\" (UID: \"8a8ad431-364b-4ffe-a252-a5dffede995e\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:19.085664 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:19.085640    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:19.218880 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:19.218859    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:19:19.221386 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:19:19.221361    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a8ad431_364b_4ffe_a252_a5dffede995e.slice/crio-4a2e434497ca2798832b08dbdd95f2112203071a3e8d0fcd0e830db63404d70b WatchSource:0}: Error finding container 4a2e434497ca2798832b08dbdd95f2112203071a3e8d0fcd0e830db63404d70b: Status 404 returned error can't find the container with id 4a2e434497ca2798832b08dbdd95f2112203071a3e8d0fcd0e830db63404d70b
Jul 13 21:19:19.534961 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:19.534884    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6" event={"ID":"8a8ad431-364b-4ffe-a252-a5dffede995e","Type":"ContainerStarted","Data":"4a2e434497ca2798832b08dbdd95f2112203071a3e8d0fcd0e830db63404d70b"}
Jul 13 21:19:19.538702 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:19.538680    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-69969d6f4c-sjv96"
Jul 13 21:19:19.593191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:19.593166    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55cc55db57-xjnx5"]
Jul 13 21:19:22.554717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:22.554677    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6" event={"ID":"8a8ad431-364b-4ffe-a252-a5dffede995e","Type":"ContainerStarted","Data":"e8699abbcb3699bd798996d3f92ac733ccd215d997eecbc2f2f125e989fc9d7a"}
Jul 13 21:19:22.555142 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:22.554814    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:22.571141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:22.571098    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6" podStartSLOduration=1.8799539950000002 podStartE2EDuration="4.571084819s" totalImagesPullingTime="2.691130824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:19:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:19:22.569692146 +0000 UTC m=+470.082441887" watchObservedRunningTime="2026-07-13 21:19:22.571084819 +0000 UTC m=+470.083834555"
Jul 13 21:19:33.561539 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:33.561505    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"
Jul 13 21:19:44.620405 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.620337    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55cc55db57-xjnx5" podUID="cb7f0b85-9a02-4872-b384-0410bd7f9195" containerName="console" containerID="cri-o://e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb" gracePeriod=15
Jul 13 21:19:44.872279 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.872216    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:19:44.976832 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.976799    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-trusted-ca-bundle\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.976859    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-service-ca\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.976904    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-serving-cert\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.976924    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-oauth-serving-cert\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.976957    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgmwj\" (UniqueName: \"kubernetes.io/projected/cb7f0b85-9a02-4872-b384-0410bd7f9195-kube-api-access-mgmwj\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.976991    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-config\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.977006    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-oauth-config\") pod \"cb7f0b85-9a02-4872-b384-0410bd7f9195\" (UID: \"cb7f0b85-9a02-4872-b384-0410bd7f9195\") "
Jul 13 21:19:44.977410 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.977339    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:19:44.977465 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.977343    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-service-ca" (OuterVolumeSpecName: "service-ca") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:19:44.977465 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.977434    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:19:44.977574 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.977508    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-config" (OuterVolumeSpecName: "console-config") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:19:44.979225 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.979198    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:19:44.979326 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.979237    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:19:44.979326 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:44.979292    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cb7f0b85-9a02-4872-b384-0410bd7f9195-kube-api-access-mgmwj" (OuterVolumeSpecName: "kube-api-access-mgmwj") pod "cb7f0b85-9a02-4872-b384-0410bd7f9195" (UID: "cb7f0b85-9a02-4872-b384-0410bd7f9195"). InnerVolumeSpecName "kube-api-access-mgmwj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:19:45.078183 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078152    2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.078183 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078180    2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-oauth-config\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.078183 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078191    2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-trusted-ca-bundle\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.078413 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078202    2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.078413 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078212    2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb7f0b85-9a02-4872-b384-0410bd7f9195-console-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.078413 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078221    2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb7f0b85-9a02-4872-b384-0410bd7f9195-oauth-serving-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.078413 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.078228    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgmwj\" (UniqueName: \"kubernetes.io/projected/cb7f0b85-9a02-4872-b384-0410bd7f9195-kube-api-access-mgmwj\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:19:45.673253 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.673218    2564 generic.go:356] "Generic (PLEG): container finished" podID="cb7f0b85-9a02-4872-b384-0410bd7f9195" containerID="e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb" exitCode=0
Jul 13 21:19:45.673631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.673260    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55cc55db57-xjnx5"
Jul 13 21:19:45.673631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.673301    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55cc55db57-xjnx5" event={"ID":"cb7f0b85-9a02-4872-b384-0410bd7f9195","Type":"ContainerDied","Data":"e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb"}
Jul 13 21:19:45.673631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.673350    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55cc55db57-xjnx5" event={"ID":"cb7f0b85-9a02-4872-b384-0410bd7f9195","Type":"ContainerDied","Data":"04f3b9e6096d017bf28edec2e19b342427d5511fe43f0b478bd5992a5fd48c55"}
Jul 13 21:19:45.673631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.673370    2564 scope.go:117] "RemoveContainer" containerID="e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb"
Jul 13 21:19:45.681879 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.681862    2564 scope.go:117] "RemoveContainer" containerID="e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb"
Jul 13 21:19:45.682158 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:19:45.682141    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb\": container with ID starting with e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb not found: ID does not exist" containerID="e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb"
Jul 13 21:19:45.682228 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.682166    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb"} err="failed to get container status \"e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb\": rpc error: code = NotFound desc = could not find container \"e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb\": container with ID starting with e528b546de2deb28fc52825c2472b132315783198d094deb989924f0531466bb not found: ID does not exist"
Jul 13 21:19:45.689388 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.689361    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55cc55db57-xjnx5"]
Jul 13 21:19:45.691241 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:45.691215    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55cc55db57-xjnx5"]
Jul 13 21:19:47.076777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:19:47.076746    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cb7f0b85-9a02-4872-b384-0410bd7f9195" path="/var/lib/kubelet/pods/cb7f0b85-9a02-4872-b384-0410bd7f9195/volumes"
Jul 13 21:20:33.151117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:33.151084    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:20:40.959479 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:40.959388    2564 generic.go:356] "Generic (PLEG): container finished" podID="e4348c7b-31e4-41e9-abc8-f780cc694d57" containerID="0f1bfcc962b88b1f100c91bfba862277660230b23551417e17c897792816100f" exitCode=0
Jul 13 21:20:40.959868 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:40.959468    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" event={"ID":"e4348c7b-31e4-41e9-abc8-f780cc694d57","Type":"ContainerDied","Data":"0f1bfcc962b88b1f100c91bfba862277660230b23551417e17c897792816100f"}
Jul 13 21:20:40.959868 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:40.959858    2564 scope.go:117] "RemoveContainer" containerID="0f1bfcc962b88b1f100c91bfba862277660230b23551417e17c897792816100f"
Jul 13 21:20:40.960930 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:40.960897    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:20:40.961001 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:40.960983    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:20:40.961064 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:40.961038    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:20:41.971566 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:41.971528    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" event={"ID":"e4348c7b-31e4-41e9-abc8-f780cc694d57","Type":"ContainerStarted","Data":"7a0c6fa96556684ce50137a081219172c7faf01bb7c3ae9febe29ac1b489b0b2"}
Jul 13 21:20:42.053935 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:42.053895    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:20:42.054149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:42.054128    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgvkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 13 21:20:42.054210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:42.054173    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" podUID="e4348c7b-31e4-41e9-abc8-f780cc694d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 13 21:20:43.053878 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:43.053839    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgvkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 13 21:20:43.054299 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:43.053899    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" podUID="e4348c7b-31e4-41e9-abc8-f780cc694d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 13 21:20:44.054300 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:44.054258    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgvkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 13 21:20:44.054677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:44.054329    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" podUID="e4348c7b-31e4-41e9-abc8-f780cc694d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 13 21:20:45.053779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:45.053744    2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vgvkp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 13 21:20:45.053963 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:45.053812    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp" podUID="e4348c7b-31e4-41e9-abc8-f780cc694d57" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 13 21:20:46.096815 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:46.096787    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:20:46.097292 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:46.097277    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:20:46.098027 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:20:46.098004    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vgvkp"
Jul 13 21:21:01.049476 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:01.049440    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:21:10.252286 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:10.252249    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:21:24.454940 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.454873    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"]
Jul 13 21:21:24.456217 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.456191    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cb7f0b85-9a02-4872-b384-0410bd7f9195" containerName="console"
Jul 13 21:21:24.456368 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.456355    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cb7f0b85-9a02-4872-b384-0410bd7f9195" containerName="console"
Jul 13 21:21:24.456593 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.456580    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cb7f0b85-9a02-4872-b384-0410bd7f9195" containerName="console"
Jul 13 21:21:24.460373 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.460348    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.462844 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.462821    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 13 21:21:24.462967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.462870    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 13 21:21:24.463828 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.463810    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-sm8dp\""
Jul 13 21:21:24.463931 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.463846    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 13 21:21:24.469490 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.469471    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"]
Jul 13 21:21:24.650287 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.650251    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06bc429a-3247-4880-9e43-3f2ea918fbd5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.650464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.650297    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzjxz\" (UniqueName: \"kubernetes.io/projected/06bc429a-3247-4880-9e43-3f2ea918fbd5-kube-api-access-pzjxz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.650464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.650318    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.650464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.650372    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.650464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.650412    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.650599 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.650522    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.751931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.751982    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06bc429a-3247-4880-9e43-3f2ea918fbd5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752010    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzjxz\" (UniqueName: \"kubernetes.io/projected/06bc429a-3247-4880-9e43-3f2ea918fbd5-kube-api-access-pzjxz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752033    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752080    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752096    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752510 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752418    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752506    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.752553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.752532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.754299 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.754267    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06bc429a-3247-4880-9e43-3f2ea918fbd5-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.754549 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.754534    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06bc429a-3247-4880-9e43-3f2ea918fbd5-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.759690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.759668    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzjxz\" (UniqueName: \"kubernetes.io/projected/06bc429a-3247-4880-9e43-3f2ea918fbd5-kube-api-access-pzjxz\") pod \"e2e-trlp-test-simulated-kserve-598466548d-622xt\" (UID: \"06bc429a-3247-4880-9e43-3f2ea918fbd5\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.773511 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.773493    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:24.915606 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:24.915571    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"]
Jul 13 21:21:24.921066 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:21:24.921027    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06bc429a_3247_4880_9e43_3f2ea918fbd5.slice/crio-d6e71c32ca611bfc4ddedf02af5d3ea18c226570f175a05aadf4d061d4a55c8a WatchSource:0}: Error finding container d6e71c32ca611bfc4ddedf02af5d3ea18c226570f175a05aadf4d061d4a55c8a: Status 404 returned error can't find the container with id d6e71c32ca611bfc4ddedf02af5d3ea18c226570f175a05aadf4d061d4a55c8a
Jul 13 21:21:25.197166 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:25.196841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt" event={"ID":"06bc429a-3247-4880-9e43-3f2ea918fbd5","Type":"ContainerStarted","Data":"d6e71c32ca611bfc4ddedf02af5d3ea18c226570f175a05aadf4d061d4a55c8a"}
Jul 13 21:21:25.853226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:25.853194    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:21:26.204509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:26.204415    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt" event={"ID":"06bc429a-3247-4880-9e43-3f2ea918fbd5","Type":"ContainerStarted","Data":"b0df1a6c016fb3deec243d3687dbbe4e8ac54f30c24a82fd47bd36f89fc7c8f4"}
Jul 13 21:21:26.227402 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:26.227345    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt" podStartSLOduration=1.193357009 podStartE2EDuration="2.227325769s" totalImagesPullingTime="1.03396876s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:21:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:21:26.225892254 +0000 UTC m=+593.738641995" watchObservedRunningTime="2026-07-13 21:21:26.227325769 +0000 UTC m=+593.740075510"
Jul 13 21:21:31.546361 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:31.546316    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:21:32.972148 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:32.972112    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:21:32.972606 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:32.972210    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:21:34.774117 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:34.774045    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:34.774585 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:34.774150    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:34.786942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:34.786910    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:35.269542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:35.269514    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-622xt"
Jul 13 21:21:51.950900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:21:51.950867    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:22:02.349153 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:22:02.349068    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:23:54.660354 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:23:54.660318    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:24:07.046171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:24:07.046136    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:24:16.148141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:24:16.148104    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:24:25.249541 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:24:25.249505    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:24:36.052659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:24:36.052627    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:25:05.955890 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:25:05.955815    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:25:18.153478 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:25:18.153443    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:25:55.356248 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:25:55.356216    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:06.774230 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:06.774193    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:15.650000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:15.649963    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:22.352749 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:22.352712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:32.855069 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:32.854969    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:33.006431 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:33.006406    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:26:33.006605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:33.006449    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:26:33.640974 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:26:33.640940    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:26:33.640974 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:26:33.640976    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:26:35.849072 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:26:35.849026    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:26:44.556379 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:44.556340    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:45.337263 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:26:45.337230    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:26:45.858946 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:26:45.858913    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:26:55.649948 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:26:55.649910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:26:55.911770 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:26:55.911676    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:00.303459 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:27:00.303420    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:05.355020 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:27:05.354984    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:27:05.921670 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:27:05.921631    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:15.341075 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:27:15.341024    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:15.931577 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:27:15.931525    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:25.983779 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:27:25.983746    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:27.953670 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:27:27.953634    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:27:30.291830 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:27:30.291793    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/system.slice/systemd-tmpfiles-clean.service\": RecentStats: unable to find data in memory cache]"
Jul 13 21:27:36.856111 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:27:36.856072    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:28:00.957128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:28:00.957033    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:28:11.349820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:28:11.349781    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:28:26.450301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:28:26.450267    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:28:36.960016 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:28:36.959978    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:28:46.353929 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:28:46.353888    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:28:55.651430 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:28:55.651394    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:29:05.957143 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:05.957110    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:29:32.151474 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:32.151382    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"]
Jul 13 21:29:32.151960 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:32.151672    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn" podUID="e16b212e-63e7-431f-9fde-e44fbd5dda47" containerName="manager" containerID="cri-o://41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04" gracePeriod=10
Jul 13 21:29:35.507465 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.507438    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:29:35.570937 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.570904    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tl64b\" (UniqueName: \"kubernetes.io/projected/e16b212e-63e7-431f-9fde-e44fbd5dda47-kube-api-access-tl64b\") pod \"e16b212e-63e7-431f-9fde-e44fbd5dda47\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") "
Jul 13 21:29:35.571168 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.570971    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e16b212e-63e7-431f-9fde-e44fbd5dda47-extensions-socket-volume\") pod \"e16b212e-63e7-431f-9fde-e44fbd5dda47\" (UID: \"e16b212e-63e7-431f-9fde-e44fbd5dda47\") "
Jul 13 21:29:35.571376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.571345    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e16b212e-63e7-431f-9fde-e44fbd5dda47-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e16b212e-63e7-431f-9fde-e44fbd5dda47" (UID: "e16b212e-63e7-431f-9fde-e44fbd5dda47"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:29:35.572902 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.572879    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e16b212e-63e7-431f-9fde-e44fbd5dda47-kube-api-access-tl64b" (OuterVolumeSpecName: "kube-api-access-tl64b") pod "e16b212e-63e7-431f-9fde-e44fbd5dda47" (UID: "e16b212e-63e7-431f-9fde-e44fbd5dda47"). InnerVolumeSpecName "kube-api-access-tl64b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:29:35.672171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.672101    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tl64b\" (UniqueName: \"kubernetes.io/projected/e16b212e-63e7-431f-9fde-e44fbd5dda47-kube-api-access-tl64b\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:29:35.672171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.672127    2564 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e16b212e-63e7-431f-9fde-e44fbd5dda47-extensions-socket-volume\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:29:35.779445 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.779410    2564 generic.go:356] "Generic (PLEG): container finished" podID="e16b212e-63e7-431f-9fde-e44fbd5dda47" containerID="41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04" exitCode=0
Jul 13 21:29:35.779609 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.779452    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"
Jul 13 21:29:35.779609 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.779459    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn" event={"ID":"e16b212e-63e7-431f-9fde-e44fbd5dda47","Type":"ContainerDied","Data":"41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04"}
Jul 13 21:29:35.779609 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.779510    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn" event={"ID":"e16b212e-63e7-431f-9fde-e44fbd5dda47","Type":"ContainerDied","Data":"2db596e881f67e66e28a2415707e2135d2820c0dff2aed76fb4882d0786f2e08"}
Jul 13 21:29:35.779609 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.779540    2564 scope.go:117] "RemoveContainer" containerID="41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04"
Jul 13 21:29:35.797662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.797639    2564 scope.go:117] "RemoveContainer" containerID="41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04"
Jul 13 21:29:35.797926 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:29:35.797905    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04\": container with ID starting with 41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04 not found: ID does not exist" containerID="41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04"
Jul 13 21:29:35.797994 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.797939    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04"} err="failed to get container status \"41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04\": rpc error: code = NotFound desc = could not find container \"41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04\": container with ID starting with 41d2c7cd57bac6bdca28b43c28cca920887501f1ff104c892e50541c676c1d04 not found: ID does not exist"
Jul 13 21:29:35.805825 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.805796    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"]
Jul 13 21:29:35.808966 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:35.808943    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-ljztn"]
Jul 13 21:29:37.076863 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:29:37.076830    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e16b212e-63e7-431f-9fde-e44fbd5dda47" path="/var/lib/kubelet/pods/e16b212e-63e7-431f-9fde-e44fbd5dda47/volumes"
Jul 13 21:30:00.145613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.145579    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"]
Jul 13 21:30:00.146025 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.146012    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e16b212e-63e7-431f-9fde-e44fbd5dda47" containerName="manager"
Jul 13 21:30:00.146025 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.146022    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e16b212e-63e7-431f-9fde-e44fbd5dda47" containerName="manager"
Jul 13 21:30:00.146131 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.146122    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e16b212e-63e7-431f-9fde-e44fbd5dda47" containerName="manager"
Jul 13 21:30:00.149258 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.149241    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"
Jul 13 21:30:00.151680 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.151654    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:30:00.155758 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.155726    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"]
Jul 13 21:30:00.161338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.161319    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"
Jul 13 21:30:00.299438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.299416    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"]
Jul 13 21:30:00.301895 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:30:00.301862    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee7005ea_fd02_4f40_942d_daf100cf1892.slice/crio-7e15bdc5fac51f45c53e4563a40a5531905d950b325311f08a9e739f98ecc97a WatchSource:0}: Error finding container 7e15bdc5fac51f45c53e4563a40a5531905d950b325311f08a9e739f98ecc97a: Status 404 returned error can't find the container with id 7e15bdc5fac51f45c53e4563a40a5531905d950b325311f08a9e739f98ecc97a
Jul 13 21:30:00.304191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.304170    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:30:00.912776 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:00.912741    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s" event={"ID":"ee7005ea-fd02-4f40-942d-daf100cf1892","Type":"ContainerStarted","Data":"7e15bdc5fac51f45c53e4563a40a5531905d950b325311f08a9e739f98ecc97a"}
Jul 13 21:30:03.930773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:03.930733    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee7005ea-fd02-4f40-942d-daf100cf1892" containerID="4e4376996ec168e3eda52d67982f7c19cc88afce01948fc7261ebcfc51b4aa21" exitCode=0
Jul 13 21:30:03.931188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:03.930819    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s" event={"ID":"ee7005ea-fd02-4f40-942d-daf100cf1892","Type":"ContainerDied","Data":"4e4376996ec168e3eda52d67982f7c19cc88afce01948fc7261ebcfc51b4aa21"}
Jul 13 21:30:05.067581 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:05.067550    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"
Jul 13 21:30:05.943074 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:05.943013    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s"
Jul 13 21:30:05.943074 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:05.943071    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29732970-2hm4s" event={"ID":"ee7005ea-fd02-4f40-942d-daf100cf1892","Type":"ContainerDied","Data":"7e15bdc5fac51f45c53e4563a40a5531905d950b325311f08a9e739f98ecc97a"}
Jul 13 21:30:05.943295 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:05.943116    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7e15bdc5fac51f45c53e4563a40a5531905d950b325311f08a9e739f98ecc97a"
Jul 13 21:30:33.321615 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.321578    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"]
Jul 13 21:30:33.322266 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.322246    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee7005ea-fd02-4f40-942d-daf100cf1892" containerName="cleanup"
Jul 13 21:30:33.322266 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.322267    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee7005ea-fd02-4f40-942d-daf100cf1892" containerName="cleanup"
Jul 13 21:30:33.322410 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.322384    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee7005ea-fd02-4f40-942d-daf100cf1892" containerName="cleanup"
Jul 13 21:30:33.325954 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.325932    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.328544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.328525    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sghhq\""
Jul 13 21:30:33.337986 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.337928    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"]
Jul 13 21:30:33.432908 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.432873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tw4b\" (UniqueName: \"kubernetes.io/projected/b701ac41-93fe-4670-9e37-d891c84c964f-kube-api-access-8tw4b\") pod \"kuadrant-operator-controller-manager-86878c769f-75dt6\" (UID: \"b701ac41-93fe-4670-9e37-d891c84c964f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.433106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.432935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b701ac41-93fe-4670-9e37-d891c84c964f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-75dt6\" (UID: \"b701ac41-93fe-4670-9e37-d891c84c964f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.533972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.533934    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8tw4b\" (UniqueName: \"kubernetes.io/projected/b701ac41-93fe-4670-9e37-d891c84c964f-kube-api-access-8tw4b\") pod \"kuadrant-operator-controller-manager-86878c769f-75dt6\" (UID: \"b701ac41-93fe-4670-9e37-d891c84c964f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.534149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.533983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b701ac41-93fe-4670-9e37-d891c84c964f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-75dt6\" (UID: \"b701ac41-93fe-4670-9e37-d891c84c964f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.534359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.534342    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b701ac41-93fe-4670-9e37-d891c84c964f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-75dt6\" (UID: \"b701ac41-93fe-4670-9e37-d891c84c964f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.542557 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.542532    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tw4b\" (UniqueName: \"kubernetes.io/projected/b701ac41-93fe-4670-9e37-d891c84c964f-kube-api-access-8tw4b\") pod \"kuadrant-operator-controller-manager-86878c769f-75dt6\" (UID: \"b701ac41-93fe-4670-9e37-d891c84c964f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.635896 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.635807    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:33.780805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:33.780781    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"]
Jul 13 21:30:33.783118 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:30:33.783093    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb701ac41_93fe_4670_9e37_d891c84c964f.slice/crio-47be5cab5b48273d1eed04e4c748b96451c17edb52dd28510e03a9c5df4ad4f7 WatchSource:0}: Error finding container 47be5cab5b48273d1eed04e4c748b96451c17edb52dd28510e03a9c5df4ad4f7: Status 404 returned error can't find the container with id 47be5cab5b48273d1eed04e4c748b96451c17edb52dd28510e03a9c5df4ad4f7
Jul 13 21:30:34.090499 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:34.090465    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6" event={"ID":"b701ac41-93fe-4670-9e37-d891c84c964f","Type":"ContainerStarted","Data":"7f6cc2130d233a0fc572f97417fad050a69669a3d8af49c44371086bb75c736a"}
Jul 13 21:30:34.090686 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:34.090533    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:34.090686 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:34.090555    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6" event={"ID":"b701ac41-93fe-4670-9e37-d891c84c964f","Type":"ContainerStarted","Data":"47be5cab5b48273d1eed04e4c748b96451c17edb52dd28510e03a9c5df4ad4f7"}
Jul 13 21:30:34.109530 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:34.109480    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6" podStartSLOduration=1.10946465 podStartE2EDuration="1.10946465s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:30:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:30:34.107224269 +0000 UTC m=+1141.619974008" watchObservedRunningTime="2026-07-13 21:30:34.10946465 +0000 UTC m=+1141.622214393"
Jul 13 21:30:45.097774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:45.097743    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-75dt6"
Jul 13 21:30:53.655400 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:30:53.655322    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:31:02.755259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:02.755215    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:31:14.562698 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:14.562664    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:31:24.911363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:24.911328    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:31:33.041354 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:33.041326    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:31:33.042296 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:33.042274    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:31:36.746007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:36.745971    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:31:46.365982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:46.365948    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:31:56.950464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:31:56.950430    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:32:06.356568 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:32:06.356529    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:32:16.853634 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:32:16.853602    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:32:26.854700 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:32:26.854607    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:32:38.154011 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:32:38.153976    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:33:00.549422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:33:00.549385    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:33:09.662066 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:33:09.662012    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:33:19.255610 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:33:19.255575    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:33:30.947825 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:33:30.947792    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:33:48.049881 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:33:48.049848    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:33:58.849184 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:33:58.849104    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:34:07.849717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:34:07.849683    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:34:18.545764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:34:18.545730    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:34:27.286575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:34:27.286538    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:34:36.752489 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:34:36.752449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:34:49.948713 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:34:49.948671    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:34:58.759798 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:34:58.759756    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:35:08.545743 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:35:08.545712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:35:21.049598 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:35:21.049555    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:35:30.244005 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:35:30.243913    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:35:36.660925 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:35:36.660890    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:35:55.251893 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:35:55.251858    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:36:08.253247 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:08.253209    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:36:20.054008 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:20.053968    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:36:32.453754 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:32.453722    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:36:33.075659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:33.075632    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:36:33.078028 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:33.078006    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:36:44.653448 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:44.653412    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:36:50.432038 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.432007    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"]
Jul 13 21:36:50.436207 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.436187    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.440385 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.440367    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-19275bdc-openshift-default-dockercfg-cg9r5\""
Jul 13 21:36:50.479596 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.479574    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"]
Jul 13 21:36:50.530863 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.530832    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dd2071a-1685-4e79-93de-d2b10f701a19-istiod-ca-cert\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.530897    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-credential-socket\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.530938    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-data\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.530960    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-podinfo\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.530985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgqng\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-kube-api-access-qgqng\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531236 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.531012    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-certs\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531236 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.531071    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-socket\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531236 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.531120    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-envoy\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.531236 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.531141    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-token\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.631923 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.631891    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-credential-socket\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.631940    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-data\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.631959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-podinfo\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.631983    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgqng\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-kube-api-access-qgqng\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632004    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-certs\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632025    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-socket\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632046    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-envoy\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632419 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632099    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-token\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632419 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632171    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dd2071a-1685-4e79-93de-d2b10f701a19-istiod-ca-cert\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632419 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632391    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-data\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632457    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-certs\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632536    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-credential-socket\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632803 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632785    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-socket\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.632873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.632825    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dd2071a-1685-4e79-93de-d2b10f701a19-istiod-ca-cert\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.634477 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.634453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-envoy\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.634588 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.634571    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-podinfo\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.641941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.641919    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgqng\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-kube-api-access-qgqng\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.647020 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.646993    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-token\") pod \"e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") " pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.747020 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.746943    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:50.886014 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.885989    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"]
Jul 13 21:36:50.887911 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:36:50.887887    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd2071a_1685_4e79_93de_d2b10f701a19.slice/crio-0f844f7be16286009b6c89e65cfc91b4cc09a552128b482419698ceb319ce71b WatchSource:0}: Error finding container 0f844f7be16286009b6c89e65cfc91b4cc09a552128b482419698ceb319ce71b: Status 404 returned error can't find the container with id 0f844f7be16286009b6c89e65cfc91b4cc09a552128b482419698ceb319ce71b
Jul 13 21:36:50.890123 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.890080    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:36:50.890216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.890165    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:36:50.890216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:50.890203    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:36:51.077963 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:51.077929    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" event={"ID":"2dd2071a-1685-4e79-93de-d2b10f701a19","Type":"ContainerStarted","Data":"dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"}
Jul 13 21:36:51.078163 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:51.077977    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" event={"ID":"2dd2071a-1685-4e79-93de-d2b10f701a19","Type":"ContainerStarted","Data":"0f844f7be16286009b6c89e65cfc91b4cc09a552128b482419698ceb319ce71b"}
Jul 13 21:36:51.100610 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:51.100557    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podStartSLOduration=1.100543988 podStartE2EDuration="1.100543988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:36:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:36:51.098924005 +0000 UTC m=+1518.611673762" watchObservedRunningTime="2026-07-13 21:36:51.100543988 +0000 UTC m=+1518.613293709"
Jul 13 21:36:51.748015 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:51.747987    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:51.753378 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:51.753356    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:52.083611 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:52.083577    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:52.084505 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:52.084488    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:36:54.851256 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:54.851223    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:36:56.706029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.705984    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"]
Jul 13 21:36:56.715077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.711531    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:36:56.715899 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.715487    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:36:56.715899 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.715746    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:36:56.716560 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.716506    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"]
Jul 13 21:36:56.791532 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.791498    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa905b6e-e89f-42a8-bbcf-8051bf020a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc\" (UID: \"fa905b6e-e89f-42a8-bbcf-8051bf020a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:36:56.892867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.892805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa905b6e-e89f-42a8-bbcf-8051bf020a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc\" (UID: \"fa905b6e-e89f-42a8-bbcf-8051bf020a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:36:56.893496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:56.893471    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa905b6e-e89f-42a8-bbcf-8051bf020a2f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc\" (UID: \"fa905b6e-e89f-42a8-bbcf-8051bf020a2f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:36:57.030925 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:57.030844    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:36:57.191614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:57.191589    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"]
Jul 13 21:36:57.193222 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:36:57.193186    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa905b6e_e89f_42a8_bbcf_8051bf020a2f.slice/crio-6e8a6f8df8873cad97c6ce28d04bc6d8c52dd9dfc4550e83f84d4693f2bc56ef WatchSource:0}: Error finding container 6e8a6f8df8873cad97c6ce28d04bc6d8c52dd9dfc4550e83f84d4693f2bc56ef: Status 404 returned error can't find the container with id 6e8a6f8df8873cad97c6ce28d04bc6d8c52dd9dfc4550e83f84d4693f2bc56ef
Jul 13 21:36:58.128867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:58.128816    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc" event={"ID":"fa905b6e-e89f-42a8-bbcf-8051bf020a2f","Type":"ContainerStarted","Data":"ec4ae70f82f9808478a16d9a3be59f0caf2a2d7f55691d8919e22165f20bf823"}
Jul 13 21:36:58.128867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:58.128872    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc" event={"ID":"fa905b6e-e89f-42a8-bbcf-8051bf020a2f","Type":"ContainerStarted","Data":"6e8a6f8df8873cad97c6ce28d04bc6d8c52dd9dfc4550e83f84d4693f2bc56ef"}
Jul 13 21:36:58.144161 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:58.144117    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc" podStartSLOduration=2.144100229 podStartE2EDuration="2.144100229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:36:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:36:58.142346984 +0000 UTC m=+1525.655096725" watchObservedRunningTime="2026-07-13 21:36:58.144100229 +0000 UTC m=+1525.656849973"
Jul 13 21:36:59.135445 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:59.135414    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerID="ec4ae70f82f9808478a16d9a3be59f0caf2a2d7f55691d8919e22165f20bf823" exitCode=7
Jul 13 21:36:59.135853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:59.135502    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc" event={"ID":"fa905b6e-e89f-42a8-bbcf-8051bf020a2f","Type":"ContainerDied","Data":"ec4ae70f82f9808478a16d9a3be59f0caf2a2d7f55691d8919e22165f20bf823"}
Jul 13 21:36:59.135853 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:36:59.135648    2564 scope.go:117] "RemoveContainer" containerID="ec4ae70f82f9808478a16d9a3be59f0caf2a2d7f55691d8919e22165f20bf823"
Jul 13 21:37:00.143076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:00.142967    2564 generic.go:356] "Generic (PLEG): container finished" podID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerID="59ba96d3afbd326e26efd32ac6f754cde553f55a56facc8e0662eadaf990a6fd" exitCode=0
Jul 13 21:37:00.143076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:00.143039    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc" event={"ID":"fa905b6e-e89f-42a8-bbcf-8051bf020a2f","Type":"ContainerDied","Data":"59ba96d3afbd326e26efd32ac6f754cde553f55a56facc8e0662eadaf990a6fd"}
Jul 13 21:37:00.143500 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:00.143157    2564 scope.go:117] "RemoveContainer" containerID="ec4ae70f82f9808478a16d9a3be59f0caf2a2d7f55691d8919e22165f20bf823"
Jul 13 21:37:01.298745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:01.298718    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:37:01.449546 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:01.449472    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa905b6e-e89f-42a8-bbcf-8051bf020a2f-maas-api-service-ca\") pod \"fa905b6e-e89f-42a8-bbcf-8051bf020a2f\" (UID: \"fa905b6e-e89f-42a8-bbcf-8051bf020a2f\") "
Jul 13 21:37:01.449849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:01.449820    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa905b6e-e89f-42a8-bbcf-8051bf020a2f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa905b6e-e89f-42a8-bbcf-8051bf020a2f" (UID: "fa905b6e-e89f-42a8-bbcf-8051bf020a2f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:37:01.551527 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:01.551483    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa905b6e-e89f-42a8-bbcf-8051bf020a2f-maas-api-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:02.157281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:02.157252    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc"
Jul 13 21:37:02.157281 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:02.157277    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc" event={"ID":"fa905b6e-e89f-42a8-bbcf-8051bf020a2f","Type":"ContainerDied","Data":"6e8a6f8df8873cad97c6ce28d04bc6d8c52dd9dfc4550e83f84d4693f2bc56ef"}
Jul 13 21:37:02.157508 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:02.157319    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6e8a6f8df8873cad97c6ce28d04bc6d8c52dd9dfc4550e83f84d4693f2bc56ef"
Jul 13 21:37:03.163992 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:03.163962    2564 generic.go:356] "Generic (PLEG): container finished" podID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerID="dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488" exitCode=0
Jul 13 21:37:03.164476 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:03.164002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" event={"ID":"2dd2071a-1685-4e79-93de-d2b10f701a19","Type":"ContainerDied","Data":"dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"}
Jul 13 21:37:03.164476 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:03.164466    2564 scope.go:117] "RemoveContainer" containerID="dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"
Jul 13 21:37:03.165508 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:03.165481    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:03.165572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:03.165550    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:03.165607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:03.165582    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:04.170792 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:04.170760    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" event={"ID":"2dd2071a-1685-4e79-93de-d2b10f701a19","Type":"ContainerStarted","Data":"e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed"}
Jul 13 21:37:04.747801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:04.747752    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:37:04.748788 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:04.748760    2564 patch_prober.go:28] interesting pod/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 13 21:37:04.748919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:04.748811    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 13 21:37:05.747595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:05.747556    2564 patch_prober.go:28] interesting pod/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 13 21:37:05.748008 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:05.747615    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 13 21:37:06.747616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:06.747577    2564 patch_prober.go:28] interesting pod/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 13 21:37:06.748067 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:06.747635    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 13 21:37:07.083477 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:07.083443    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:37:07.747965 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:07.747930    2564 patch_prober.go:28] interesting pod/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused" start-of-body=
Jul 13 21:37:07.748386 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:07.747983    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.51:15021/healthz/ready\": dial tcp 10.134.0.51:15021: connect: connection refused"
Jul 13 21:37:08.752042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:08.752008    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:37:08.752985 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:08.752965    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:37:12.156719 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:12.156681    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:37:21.969517 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:21.969479    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"]
Jul 13 21:37:21.970046 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:21.969687    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy" containerID="cri-o://e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed" gracePeriod=30
Jul 13 21:37:22.562090 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.562040    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"]
Jul 13 21:37:22.562703 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.562670    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerName="revoke-keys"
Jul 13 21:37:22.562849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.562736    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerName="revoke-keys"
Jul 13 21:37:22.562922 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.562893    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerName="revoke-keys"
Jul 13 21:37:22.562922 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.562914    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerName="revoke-keys"
Jul 13 21:37:22.566358 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.566337    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.569119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.569097    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-737411d0-gw-openshift-default-dockercfg-tpgpz\""
Jul 13 21:37:22.581303 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.581279    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"]
Jul 13 21:37:22.662312 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-token\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662329    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-credential-socket\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662383    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-podinfo\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662416    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-socket\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662470    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-envoy\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662515    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-data\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662532    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5fb826-e792-466e-97a4-2c605ede0ba8-istiod-ca-cert\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662595 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662556    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qxv6\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-kube-api-access-4qxv6\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.662729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.662605    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-certs\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763655    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-credential-socket\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-podinfo\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763734    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-socket\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763753    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-envoy\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763782    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-data\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763797    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5fb826-e792-466e-97a4-2c605ede0ba8-istiod-ca-cert\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763822    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qxv6\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-kube-api-access-4qxv6\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763893    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-certs\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.763957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.763942    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-token\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.764449 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.764119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-credential-socket\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.764449 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.764181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-socket\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.764449 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.764272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-data\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.764449 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.764394    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-certs\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.764690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.764666    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5fb826-e792-466e-97a4-2c605ede0ba8-istiod-ca-cert\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.766139 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.766121    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-envoy\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.766417 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.766401    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-podinfo\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.772365 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.772346    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-token\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.772673 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.772649    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qxv6\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-kube-api-access-4qxv6\") pod \"e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") " pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:22.877338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:22.877306    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:23.019567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.019469    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"]
Jul 13 21:37:23.022021 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:37:23.021992    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee5fb826_e792_466e_97a4_2c605ede0ba8.slice/crio-a90b18e5d8ec814dd225a9ac8901746e380db102eccebb97e5f5dad3632585f9 WatchSource:0}: Error finding container a90b18e5d8ec814dd225a9ac8901746e380db102eccebb97e5f5dad3632585f9: Status 404 returned error can't find the container with id a90b18e5d8ec814dd225a9ac8901746e380db102eccebb97e5f5dad3632585f9
Jul 13 21:37:23.024206 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.024173    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:23.024273 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.024241    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:23.024308 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.024270    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:23.287201 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.287109    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" event={"ID":"ee5fb826-e792-466e-97a4-2c605ede0ba8","Type":"ContainerStarted","Data":"a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"}
Jul 13 21:37:23.287201 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.287167    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" event={"ID":"ee5fb826-e792-466e-97a4-2c605ede0ba8","Type":"ContainerStarted","Data":"a90b18e5d8ec814dd225a9ac8901746e380db102eccebb97e5f5dad3632585f9"}
Jul 13 21:37:23.305682 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.305629    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podStartSLOduration=1.30561162 podStartE2EDuration="1.30561162s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:23.30440297 +0000 UTC m=+1550.817152712" watchObservedRunningTime="2026-07-13 21:37:23.30561162 +0000 UTC m=+1550.818361361"
Jul 13 21:37:23.754926 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.754893    2564 patch_prober.go:28] interesting pod/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 21:37:23.755119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.754948    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:37:23.878369 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:23.878338    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:24.883108 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:24.883075    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:25.298799 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:25.298710    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:25.299845 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:25.299823    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:25.955724 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:25.955687    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:37:27.256958 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.256930    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324107    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-socket\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324156    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-envoy\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324205    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-certs\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324271    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dd2071a-1685-4e79-93de-d2b10f701a19-istiod-ca-cert\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324300    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-token\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324374    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgqng\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-kube-api-access-qgqng\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324412    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-podinfo\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324444    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-data\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.325121 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.324499    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-credential-socket\") pod \"2dd2071a-1685-4e79-93de-d2b10f701a19\" (UID: \"2dd2071a-1685-4e79-93de-d2b10f701a19\") "
Jul 13 21:37:27.329227 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.325785    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:27.329227 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.328786    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:27.330149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.330111    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.330464    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dd2071a-1685-4e79-93de-d2b10f701a19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.330708    2564 generic.go:356] "Generic (PLEG): container finished" podID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerID="e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed" exitCode=0
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.330969    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-data" (OuterVolumeSpecName: "istio-data") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.331073    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.331408    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" event={"ID":"2dd2071a-1685-4e79-93de-d2b10f701a19","Type":"ContainerDied","Data":"e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed"}
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.331454    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv" event={"ID":"2dd2071a-1685-4e79-93de-d2b10f701a19","Type":"ContainerDied","Data":"0f844f7be16286009b6c89e65cfc91b4cc09a552128b482419698ceb319ce71b"}
Jul 13 21:37:27.331607 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.331482    2564 scope.go:117] "RemoveContainer" containerID="e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed"
Jul 13 21:37:27.333226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.333036    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:37:27.333309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.333245    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-kube-api-access-qgqng" (OuterVolumeSpecName: "kube-api-access-qgqng") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "kube-api-access-qgqng". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:37:27.335200 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.335166    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:27.335548 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.335519    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-token" (OuterVolumeSpecName: "istio-token") pod "2dd2071a-1685-4e79-93de-d2b10f701a19" (UID: "2dd2071a-1685-4e79-93de-d2b10f701a19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:37:27.355502 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.355477    2564 scope.go:117] "RemoveContainer" containerID="dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"
Jul 13 21:37:27.365259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.365238    2564 scope.go:117] "RemoveContainer" containerID="e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed"
Jul 13 21:37:27.365565 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:37:27.365539    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed\": container with ID starting with e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed not found: ID does not exist" containerID="e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed"
Jul 13 21:37:27.365656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.365577    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed"} err="failed to get container status \"e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed\": rpc error: code = NotFound desc = could not find container \"e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed\": container with ID starting with e0c80012899dcf5f401e087a209d22c67e485a03daed4157d66d495d0341e8ed not found: ID does not exist"
Jul 13 21:37:27.365656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.365603    2564 scope.go:117] "RemoveContainer" containerID="dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"
Jul 13 21:37:27.365979 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:37:27.365869    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488\": container with ID starting with dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488 not found: ID does not exist" containerID="dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"
Jul 13 21:37:27.365979 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.365898    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488"} err="failed to get container status \"dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488\": rpc error: code = NotFound desc = could not find container \"dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488\": container with ID starting with dae7022b2e7e8e0bee940de4a4245fa022aff51e92dc5cd5791e411032bb4488 not found: ID does not exist"
Jul 13 21:37:27.426103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.425992    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2dd2071a-1685-4e79-93de-d2b10f701a19-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426025    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426038    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgqng\" (UniqueName: \"kubernetes.io/projected/2dd2071a-1685-4e79-93de-d2b10f701a19-kube-api-access-qgqng\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426087    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426103    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426116    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426131    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426144    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.426450 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.426156    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2dd2071a-1685-4e79-93de-d2b10f701a19-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:27.653039 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.653013    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"]
Jul 13 21:37:27.656588 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:27.656562    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-19275bdc-openshift-default-547c86fdf7-jrmwv"]
Jul 13 21:37:29.077422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:29.077381    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" path="/var/lib/kubelet/pods/2dd2071a-1685-4e79-93de-d2b10f701a19/volumes"
Jul 13 21:37:34.373726 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:34.373692    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerID="a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26" exitCode=0
Jul 13 21:37:34.374172 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:34.373738    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" event={"ID":"ee5fb826-e792-466e-97a4-2c605ede0ba8","Type":"ContainerDied","Data":"a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"}
Jul 13 21:37:34.374222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:34.374196    2564 scope.go:117] "RemoveContainer" containerID="a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"
Jul 13 21:37:34.375325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:34.375293    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:34.375404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:34.375364    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:34.375404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:34.375395    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:35.381956 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:35.381922    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" event={"ID":"ee5fb826-e792-466e-97a4-2c605ede0ba8","Type":"ContainerStarted","Data":"986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7"}
Jul 13 21:37:35.877782 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:35.877753    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:35.879066 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:35.879026    2564 patch_prober.go:28] interesting pod/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 13 21:37:35.879141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:35.879102    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 13 21:37:36.878412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:36.878366    2564 patch_prober.go:28] interesting pod/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 13 21:37:36.878820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:36.878432    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 13 21:37:37.878234 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:37.878193    2564 patch_prober.go:28] interesting pod/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 13 21:37:37.878422 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:37.878254    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 13 21:37:38.878603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:38.878565    2564 patch_prober.go:28] interesting pod/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Jul 13 21:37:38.879000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:38.878622    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Jul 13 21:37:39.883023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:39.882983    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:39.883510 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:39.883318    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:39.883995 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:39.883981    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:42.760963 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:42.760922    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:37:53.602690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.602650    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"]
Jul 13 21:37:53.603196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.602858    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy" containerID="cri-o://986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7" gracePeriod=30
Jul 13 21:37:53.903272 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903193    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"]
Jul 13 21:37:53.903681 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903668    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerName="revoke-keys"
Jul 13 21:37:53.903729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903683    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa905b6e-e89f-42a8-bbcf-8051bf020a2f" containerName="revoke-keys"
Jul 13 21:37:53.903729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903693    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy"
Jul 13 21:37:53.903729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903698    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy"
Jul 13 21:37:53.903729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903712    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy"
Jul 13 21:37:53.903729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903718    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy"
Jul 13 21:37:53.903904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903782    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy"
Jul 13 21:37:53.903904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.903792    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dd2071a-1685-4e79-93de-d2b10f701a19" containerName="istio-proxy"
Jul 13 21:37:53.907224 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.907199    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.909476 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.909447    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5278c205-openshift-default-dockercfg-bvpzq\""
Jul 13 21:37:53.916736 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.916708    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"]
Jul 13 21:37:53.977263 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977237    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-socket\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977428 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977266    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-data\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977428 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977286    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-credential-socket\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977428 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977325    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf64552-0164-4631-af88-32d16a167483-istio-podinfo\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977451    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn8q6\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-kube-api-access-dn8q6\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977519    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf64552-0164-4631-af88-32d16a167483-istiod-ca-cert\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977551    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-envoy\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977583    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-certs\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:53.977763 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:53.977607    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-istio-token\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.078879 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078846    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn8q6\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-kube-api-access-dn8q6\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078890    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf64552-0164-4631-af88-32d16a167483-istiod-ca-cert\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078911    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-envoy\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078931    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-certs\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078946    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-istio-token\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078964    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-socket\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.078981    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-data\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079017    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-credential-socket\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079538 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079116    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf64552-0164-4631-af88-32d16a167483-istio-podinfo\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079538 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079423    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-certs\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079538 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-socket\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079700 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079547    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-data\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079700 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079592    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-credential-socket\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.079945 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.079924    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf64552-0164-4631-af88-32d16a167483-istiod-ca-cert\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.081284 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.081266    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-envoy\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.081553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.081538    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf64552-0164-4631-af88-32d16a167483-istio-podinfo\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.086418 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.086396    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-istio-token\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.086508 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.086437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn8q6\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-kube-api-access-dn8q6\") pod \"e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") " pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.221587 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.221513    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:54.363101 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.363043    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"]
Jul 13 21:37:54.364572 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:37:54.364545    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bf64552_0164_4631_af88_32d16a167483.slice/crio-5889e0b762a7e063cb05fe5c2c6a763341657e329e97f63c05315aa1b18ecf8d WatchSource:0}: Error finding container 5889e0b762a7e063cb05fe5c2c6a763341657e329e97f63c05315aa1b18ecf8d: Status 404 returned error can't find the container with id 5889e0b762a7e063cb05fe5c2c6a763341657e329e97f63c05315aa1b18ecf8d
Jul 13 21:37:54.366729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.366691    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:54.366838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.366763    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:54.366838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.366791    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:37:54.494944 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.494847    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" event={"ID":"9bf64552-0164-4631-af88-32d16a167483","Type":"ContainerStarted","Data":"9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"}
Jul 13 21:37:54.494944 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.494913    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" event={"ID":"9bf64552-0164-4631-af88-32d16a167483","Type":"ContainerStarted","Data":"5889e0b762a7e063cb05fe5c2c6a763341657e329e97f63c05315aa1b18ecf8d"}
Jul 13 21:37:54.515656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.515592    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podStartSLOduration=1.515571028 podStartE2EDuration="1.515571028s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:37:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:37:54.512367542 +0000 UTC m=+1582.025117282" watchObservedRunningTime="2026-07-13 21:37:54.515571028 +0000 UTC m=+1582.028320771"
Jul 13 21:37:54.885841 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.885807    2564 patch_prober.go:28] interesting pod/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 13 21:37:54.886313 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:54.885862    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:37:55.222293 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:55.222216    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:55.226950 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:55.226923    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:55.502718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:55.502629    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:55.503613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:55.503588    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:37:57.653730 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:57.653699    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:37:58.899026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.898318    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923107    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-envoy\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923196    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-certs\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923239    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-podinfo\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923267    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-token\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923328    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-data\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923360    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-socket\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923416    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-credential-socket\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923482    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5fb826-e792-466e-97a4-2c605ede0ba8-istiod-ca-cert\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.924381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.923536    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4qxv6\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-kube-api-access-4qxv6\") pod \"ee5fb826-e792-466e-97a4-2c605ede0ba8\" (UID: \"ee5fb826-e792-466e-97a4-2c605ede0ba8\") "
Jul 13 21:37:58.929677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.927774    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:58.929677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.928507    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:58.929677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.928951    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:58.929677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.929128    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ee5fb826-e792-466e-97a4-2c605ede0ba8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:37:58.929677 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.929640    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-data" (OuterVolumeSpecName: "istio-data") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:58.932363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.932321    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:37:58.932363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.932352    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:37:58.933099 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.933069    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-kube-api-access-4qxv6" (OuterVolumeSpecName: "kube-api-access-4qxv6") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "kube-api-access-4qxv6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:37:58.933909 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:58.933872    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-token" (OuterVolumeSpecName: "istio-token") pod "ee5fb826-e792-466e-97a4-2c605ede0ba8" (UID: "ee5fb826-e792-466e-97a4-2c605ede0ba8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:37:59.025645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025594    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025630    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025640    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025645 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025653    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025989 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025665    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025989 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025674    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025989 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025683    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ee5fb826-e792-466e-97a4-2c605ede0ba8-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025989 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025692    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4qxv6\" (UniqueName: \"kubernetes.io/projected/ee5fb826-e792-466e-97a4-2c605ede0ba8-kube-api-access-4qxv6\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.025989 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.025701    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ee5fb826-e792-466e-97a4-2c605ede0ba8-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:37:59.528947 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.528912    2564 generic.go:356] "Generic (PLEG): container finished" podID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerID="986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7" exitCode=0
Jul 13 21:37:59.529120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.528976    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"
Jul 13 21:37:59.529120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.528993    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" event={"ID":"ee5fb826-e792-466e-97a4-2c605ede0ba8","Type":"ContainerDied","Data":"986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7"}
Jul 13 21:37:59.529120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.529046    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt" event={"ID":"ee5fb826-e792-466e-97a4-2c605ede0ba8","Type":"ContainerDied","Data":"a90b18e5d8ec814dd225a9ac8901746e380db102eccebb97e5f5dad3632585f9"}
Jul 13 21:37:59.529120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.529089    2564 scope.go:117] "RemoveContainer" containerID="986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7"
Jul 13 21:37:59.538041 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.538024    2564 scope.go:117] "RemoveContainer" containerID="a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"
Jul 13 21:37:59.546553 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.546533    2564 scope.go:117] "RemoveContainer" containerID="986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7"
Jul 13 21:37:59.546981 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:37:59.546926    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7\": container with ID starting with 986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7 not found: ID does not exist" containerID="986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7"
Jul 13 21:37:59.546981 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.546962    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7"} err="failed to get container status \"986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7\": rpc error: code = NotFound desc = could not find container \"986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7\": container with ID starting with 986b47bcb7c2e5dc84dcee2a8aeb00fe2e14c1848dd929ba25e437e3af5adfc7 not found: ID does not exist"
Jul 13 21:37:59.546981 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.546992    2564 scope.go:117] "RemoveContainer" containerID="a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"
Jul 13 21:37:59.547284 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:37:59.547265    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26\": container with ID starting with a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26 not found: ID does not exist" containerID="a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"
Jul 13 21:37:59.547325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.547291    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26"} err="failed to get container status \"a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26\": rpc error: code = NotFound desc = could not find container \"a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26\": container with ID starting with a18524d7ee271fed615b1fb158e5dfafddf14f591e284d6c7078a4dec81fbe26 not found: ID does not exist"
Jul 13 21:37:59.549502 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.549481    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"]
Jul 13 21:37:59.551597 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:37:59.551580    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-737411d0-gw-openshift-default-64f797f8f5-88xqt"]
Jul 13 21:38:01.076662 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:01.076629    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" path="/var/lib/kubelet/pods/ee5fb826-e792-466e-97a4-2c605ede0ba8/volumes"
Jul 13 21:38:06.571758 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:06.571720    2564 generic.go:356] "Generic (PLEG): container finished" podID="9bf64552-0164-4631-af88-32d16a167483" containerID="9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e" exitCode=0
Jul 13 21:38:06.572204 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:06.571791    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" event={"ID":"9bf64552-0164-4631-af88-32d16a167483","Type":"ContainerDied","Data":"9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"}
Jul 13 21:38:06.572252 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:06.572223    2564 scope.go:117] "RemoveContainer" containerID="9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"
Jul 13 21:38:06.573255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:06.573222    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:06.573420 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:06.573308    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:06.573420 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:06.573359    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:07.579675 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:07.579641    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" event={"ID":"9bf64552-0164-4631-af88-32d16a167483","Type":"ContainerStarted","Data":"0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52"}
Jul 13 21:38:08.222271 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:08.222221    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:38:08.223357 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:08.223329    2564 patch_prober.go:28] interesting pod/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 13 21:38:08.223471 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:08.223389    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 13 21:38:09.222249 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:09.222208    2564 patch_prober.go:28] interesting pod/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 13 21:38:09.222881 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:09.222825    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 13 21:38:10.222540 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:10.222496    2564 patch_prober.go:28] interesting pod/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 13 21:38:10.222949 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:10.222571    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 13 21:38:10.502795 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:10.502684    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:38:11.223019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:11.222670    2564 patch_prober.go:28] interesting pod/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused" start-of-body=
Jul 13 21:38:11.223019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:11.222723    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.54:15021/healthz/ready\": dial tcp 10.134.0.54:15021: connect: connection refused"
Jul 13 21:38:12.225994 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:12.225956    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:38:12.227024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:12.227005    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:38:14.952741 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:14.952697    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:38:26.364911 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.364829    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"]
Jul 13 21:38:26.365300 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.365121    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy" containerID="cri-o://0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52" gracePeriod=30
Jul 13 21:38:26.655477 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.655395    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"]
Jul 13 21:38:26.655973 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.655953    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy"
Jul 13 21:38:26.656111 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.655976    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy"
Jul 13 21:38:26.656111 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.656007    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy"
Jul 13 21:38:26.656111 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.656016    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy"
Jul 13 21:38:26.656285 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.656142    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy"
Jul 13 21:38:26.656285 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.656156    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee5fb826-e792-466e-97a4-2c605ede0ba8" containerName="istio-proxy"
Jul 13 21:38:26.660816 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.660791    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.663490 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.663470    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-f9b92c93-openshift-default-dockercfg-cdsjh\""
Jul 13 21:38:26.671834 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.671812    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"]
Jul 13 21:38:26.702803 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.702777    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-socket\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.702930 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.702820    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-token\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.702930 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.702843    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-envoy\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.703103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.702928    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e992e54-f16c-455a-947d-c65dbefc5ea5-istiod-ca-cert\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.703103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.702971    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-certs\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.703103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.703077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-data\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.703269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.703118    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-credential-socket\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.703269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.703148    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrxsk\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-kube-api-access-vrxsk\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.703269 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.703166    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-podinfo\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804221 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804182    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e992e54-f16c-455a-947d-c65dbefc5ea5-istiod-ca-cert\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804221 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804222    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-certs\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804251    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-data\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804274    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-credential-socket\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804309    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrxsk\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-kube-api-access-vrxsk\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804334    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-podinfo\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804392    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-socket\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804459 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804453    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-token\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804487    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-envoy\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804638    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-certs\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804777 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804729    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-credential-socket\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.804948 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.804888    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-socket\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.805047 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.805027    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-data\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.805257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.805240    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e992e54-f16c-455a-947d-c65dbefc5ea5-istiod-ca-cert\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.806619 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.806597    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-envoy\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.806883 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.806863    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-podinfo\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.812159 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.812140    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-token\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.812524 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.812505    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrxsk\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-kube-api-access-vrxsk\") pod \"e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") " pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:26.973977 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:26.973924    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:27.115381 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.115359    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"]
Jul 13 21:38:27.117734 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:38:27.117706    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e992e54_f16c_455a_947d_c65dbefc5ea5.slice/crio-948da3dd2f958867362de7ab3b515516b2e088609278a77f90e8990d39cea629 WatchSource:0}: Error finding container 948da3dd2f958867362de7ab3b515516b2e088609278a77f90e8990d39cea629: Status 404 returned error can't find the container with id 948da3dd2f958867362de7ab3b515516b2e088609278a77f90e8990d39cea629
Jul 13 21:38:27.119933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.119896    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:27.120024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.119971    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:27.120024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.120015    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:27.228259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.228192    2564 patch_prober.go:28] interesting pod/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 13 21:38:27.228379 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.228264    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:38:27.691071 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.691022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" event={"ID":"0e992e54-f16c-455a-947d-c65dbefc5ea5","Type":"ContainerStarted","Data":"642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"}
Jul 13 21:38:27.691517 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.691089    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" event={"ID":"0e992e54-f16c-455a-947d-c65dbefc5ea5","Type":"ContainerStarted","Data":"948da3dd2f958867362de7ab3b515516b2e088609278a77f90e8990d39cea629"}
Jul 13 21:38:27.712363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.712311    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" podStartSLOduration=1.712295076 podStartE2EDuration="1.712295076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:38:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:38:27.710449791 +0000 UTC m=+1615.223199533" watchObservedRunningTime="2026-07-13 21:38:27.712295076 +0000 UTC m=+1615.225044817"
Jul 13 21:38:27.974192 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:27.974102    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:28.979842 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:28.979805    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:29.704682 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:29.704647    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:29.705583 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:29.705557    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:30.351750 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:30.351717    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:38:31.657802 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.657779    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:38:31.732076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.731084    2564 generic.go:356] "Generic (PLEG): container finished" podID="9bf64552-0164-4631-af88-32d16a167483" containerID="0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52" exitCode=0
Jul 13 21:38:31.732076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.731958    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"
Jul 13 21:38:31.732361 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.732155    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" event={"ID":"9bf64552-0164-4631-af88-32d16a167483","Type":"ContainerDied","Data":"0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52"}
Jul 13 21:38:31.732361 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.732223    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr" event={"ID":"9bf64552-0164-4631-af88-32d16a167483","Type":"ContainerDied","Data":"5889e0b762a7e063cb05fe5c2c6a763341657e329e97f63c05315aa1b18ecf8d"}
Jul 13 21:38:31.732361 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.732255    2564 scope.go:117] "RemoveContainer" containerID="0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52"
Jul 13 21:38:31.744407 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.744376    2564 scope.go:117] "RemoveContainer" containerID="9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"
Jul 13 21:38:31.755115 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.755086    2564 scope.go:117] "RemoveContainer" containerID="0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52"
Jul 13 21:38:31.755404 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:38:31.755383    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52\": container with ID starting with 0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52 not found: ID does not exist" containerID="0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52"
Jul 13 21:38:31.755491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.755418    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52"} err="failed to get container status \"0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52\": rpc error: code = NotFound desc = could not find container \"0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52\": container with ID starting with 0bd357ae1c82fa8c6091d8c81197192eb2b75db574b308e24d66b8cfc31e8d52 not found: ID does not exist"
Jul 13 21:38:31.755491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.755443    2564 scope.go:117] "RemoveContainer" containerID="9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"
Jul 13 21:38:31.755701 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:38:31.755678    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e\": container with ID starting with 9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e not found: ID does not exist" containerID="9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"
Jul 13 21:38:31.755751 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.755708    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e"} err="failed to get container status \"9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e\": rpc error: code = NotFound desc = could not find container \"9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e\": container with ID starting with 9cf7d755e59c9c97be023476eedb46ad61f8e7254775a7c44c88831425f1e07e not found: ID does not exist"
Jul 13 21:38:31.757171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757151    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-certs\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.757225 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757210    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf64552-0164-4631-af88-32d16a167483-istiod-ca-cert\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.757258 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757246    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf64552-0164-4631-af88-32d16a167483-istio-podinfo\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.757359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757342    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-data\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.757427 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757393    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:38:31.757485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757411    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-istio-token\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.757485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757464    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn8q6\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-kube-api-access-dn8q6\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.757614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757591    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9bf64552-0164-4631-af88-32d16a167483-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:38:31.757830 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757812    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-credential-socket\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.758001 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.757986    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-socket\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.758214 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.758201    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-envoy\") pod \"9bf64552-0164-4631-af88-32d16a167483\" (UID: \"9bf64552-0164-4631-af88-32d16a167483\") "
Jul 13 21:38:31.758383 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.758146    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:38:31.758463 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.758295    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:38:31.758463 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.758349    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-data" (OuterVolumeSpecName: "istio-data") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:38:31.759024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759006    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9bf64552-0164-4631-af88-32d16a167483-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.759149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759137    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.759247 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759235    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.759336 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759325    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.759429 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759407    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.759919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759894    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9bf64552-0164-4631-af88-32d16a167483-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:38:31.760000 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.759929    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-istio-token" (OuterVolumeSpecName: "istio-token") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:38:31.760198 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.760174    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-kube-api-access-dn8q6" (OuterVolumeSpecName: "kube-api-access-dn8q6") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "kube-api-access-dn8q6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:38:31.760346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.760332    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9bf64552-0164-4631-af88-32d16a167483" (UID: "9bf64552-0164-4631-af88-32d16a167483"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:38:31.860888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.860843    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn8q6\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-kube-api-access-dn8q6\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.860888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.860881    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9bf64552-0164-4631-af88-32d16a167483-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.860888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.860894    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9bf64552-0164-4631-af88-32d16a167483-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:31.861183 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:31.860906    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9bf64552-0164-4631-af88-32d16a167483-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:38:32.056203 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:32.056171    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"]
Jul 13 21:38:32.059685 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:32.059658    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5278c205-openshift-default-56d76f6555-xdrmr"]
Jul 13 21:38:33.077400 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:33.077363    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bf64552-0164-4631-af88-32d16a167483" path="/var/lib/kubelet/pods/9bf64552-0164-4631-af88-32d16a167483/volumes"
Jul 13 21:38:38.773070 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:38.773025    2564 generic.go:356] "Generic (PLEG): container finished" podID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerID="642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d" exitCode=0
Jul 13 21:38:38.773514 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:38.773084    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" event={"ID":"0e992e54-f16c-455a-947d-c65dbefc5ea5","Type":"ContainerDied","Data":"642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"}
Jul 13 21:38:38.773514 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:38.773466    2564 scope.go:117] "RemoveContainer" containerID="642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"
Jul 13 21:38:38.774464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:38.774434    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:38.774531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:38.774508    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:38.774567 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:38.774539    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:38:39.780106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:39.780070    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" event={"ID":"0e992e54-f16c-455a-947d-c65dbefc5ea5","Type":"ContainerStarted","Data":"f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb"}
Jul 13 21:38:39.974485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:39.974449    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:39.975737 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:39.975711    2564 patch_prober.go:28] interesting pod/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 13 21:38:39.975869 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:39.975757    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 13 21:38:40.975106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:40.975045    2564 patch_prober.go:28] interesting pod/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 13 21:38:40.975561 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:40.975129    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 13 21:38:41.975125 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:41.975090    2564 patch_prober.go:28] interesting pod/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 13 21:38:41.975503 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:41.975145    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 13 21:38:42.974736 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:42.974699    2564 patch_prober.go:28] interesting pod/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused" start-of-body=
Jul 13 21:38:42.974911 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:42.974757    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.55:15021/healthz/ready\": dial tcp 10.134.0.55:15021: connect: connection refused"
Jul 13 21:38:43.978184 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:43.978157    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:43.978625 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:43.978551    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:38:43.979151 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:38:43.979132    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:39:35.765867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:35.765824    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:39:47.520650 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:47.520620    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"]
Jul 13 21:39:47.521105 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:47.520838    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy" containerID="cri-o://f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb" gracePeriod=30
Jul 13 21:39:52.769473 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.769449    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:39:52.828132 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828092    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-credential-socket\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828135    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrxsk\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-kube-api-access-vrxsk\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828211    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-socket\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828237    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e992e54-f16c-455a-947d-c65dbefc5ea5-istiod-ca-cert\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828278    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-token\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828503 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828335    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-data\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828503 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828371    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-certs\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828503 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828448    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-podinfo\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828503 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828476    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-envoy\") pod \"0e992e54-f16c-455a-947d-c65dbefc5ea5\" (UID: \"0e992e54-f16c-455a-947d-c65dbefc5ea5\") "
Jul 13 21:39:52.828698 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828585    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:39:52.828936 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828827    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.829024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.828964    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:39:52.829252 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.829228    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:39:52.829499 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.829477    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e992e54-f16c-455a-947d-c65dbefc5ea5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:39:52.829648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.829629    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:39:52.831663 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.831630    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:39:52.831663 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.831653    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:39:52.831927 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.831900    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-kube-api-access-vrxsk" (OuterVolumeSpecName: "kube-api-access-vrxsk") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "kube-api-access-vrxsk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:39:52.832570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.832547    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e992e54-f16c-455a-947d-c65dbefc5ea5" (UID: "0e992e54-f16c-455a-947d-c65dbefc5ea5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:39:52.930034 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930011    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930034 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930036    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e992e54-f16c-455a-947d-c65dbefc5ea5-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930045    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930074    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930082    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930090    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930097    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e992e54-f16c-455a-947d-c65dbefc5ea5-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:52.930193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:52.930105    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrxsk\" (UniqueName: \"kubernetes.io/projected/0e992e54-f16c-455a-947d-c65dbefc5ea5-kube-api-access-vrxsk\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:39:53.176608 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.176578    2564 generic.go:356] "Generic (PLEG): container finished" podID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerID="f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb" exitCode=0
Jul 13 21:39:53.176718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.176622    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"
Jul 13 21:39:53.176718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.176658    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" event={"ID":"0e992e54-f16c-455a-947d-c65dbefc5ea5","Type":"ContainerDied","Data":"f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb"}
Jul 13 21:39:53.176718 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.176706    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb" event={"ID":"0e992e54-f16c-455a-947d-c65dbefc5ea5","Type":"ContainerDied","Data":"948da3dd2f958867362de7ab3b515516b2e088609278a77f90e8990d39cea629"}
Jul 13 21:39:53.176834 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.176737    2564 scope.go:117] "RemoveContainer" containerID="f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb"
Jul 13 21:39:53.185805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.185792    2564 scope.go:117] "RemoveContainer" containerID="642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"
Jul 13 21:39:53.194003 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.193987    2564 scope.go:117] "RemoveContainer" containerID="f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb"
Jul 13 21:39:53.194260 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:39:53.194243    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb\": container with ID starting with f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb not found: ID does not exist" containerID="f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb"
Jul 13 21:39:53.194342 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.194271    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb"} err="failed to get container status \"f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb\": rpc error: code = NotFound desc = could not find container \"f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb\": container with ID starting with f8477641722f13400f67bf5c969f0174460e23c567fbff5d03f20cf6cc691cfb not found: ID does not exist"
Jul 13 21:39:53.194342 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.194293    2564 scope.go:117] "RemoveContainer" containerID="642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"
Jul 13 21:39:53.194551 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:39:53.194533    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d\": container with ID starting with 642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d not found: ID does not exist" containerID="642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"
Jul 13 21:39:53.194601 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.194559    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d"} err="failed to get container status \"642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d\": rpc error: code = NotFound desc = could not find container \"642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d\": container with ID starting with 642ef4a91f5dd5bcd039f3560910293894fec6b800c954182720ba48c631b66d not found: ID does not exist"
Jul 13 21:39:53.195035 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.195011    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"]
Jul 13 21:39:53.199047 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:53.199015    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-f9b92c93-openshift-default-59788d77f9-bzpcb"]
Jul 13 21:39:55.077033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:39:55.077000    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" path="/var/lib/kubelet/pods/0e992e54-f16c-455a-947d-c65dbefc5ea5/volumes"
Jul 13 21:41:16.752572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:16.752539    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:41:25.654219 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:25.654140    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:41:29.052949 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.052911    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"]
Jul 13 21:41:29.053764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053738    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy"
Jul 13 21:41:29.053764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053764    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy"
Jul 13 21:41:29.053934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053777    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy"
Jul 13 21:41:29.053934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053786    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy"
Jul 13 21:41:29.053934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053910    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy"
Jul 13 21:41:29.053934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053928    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy"
Jul 13 21:41:29.054166 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.053938    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy"
Jul 13 21:41:29.057162 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.057141    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.059584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.059566    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2b4e0142-openshift-default-dockercfg-dwljd\""
Jul 13 21:41:29.078837 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.078806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"]
Jul 13 21:41:29.132204 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132173    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-socket\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132211    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-certs\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132245    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517011c0-91dc-43cc-a620-ad73779e5172-istiod-ca-cert\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132280    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-envoy\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132302    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-data\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132322    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517011c0-91dc-43cc-a620-ad73779e5172-istio-podinfo\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-credential-socket\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-istio-token\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.132509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.132437    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbgwv\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-kube-api-access-kbgwv\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233597 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233558    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-socket\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233608    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-certs\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517011c0-91dc-43cc-a620-ad73779e5172-istiod-ca-cert\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-envoy\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233686    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-data\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233710    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517011c0-91dc-43cc-a620-ad73779e5172-istio-podinfo\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.233765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233736    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-credential-socket\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233787    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-istio-token\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233845    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbgwv\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-kube-api-access-kbgwv\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234088 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.233990    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-socket\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.234127    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-certs\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.234209    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-data\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.234372    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517011c0-91dc-43cc-a620-ad73779e5172-istiod-ca-cert\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.234507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.234472    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-credential-socket\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.236031 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.236006    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-envoy\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.236246 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.236228    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517011c0-91dc-43cc-a620-ad73779e5172-istio-podinfo\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.241411 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.241389    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-istio-token\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.241520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.241437    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbgwv\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-kube-api-access-kbgwv\") pod \"e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") " pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.368264 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.368231    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:29.505958 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.505933    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"]
Jul 13 21:41:29.507817 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:41:29.507791    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod517011c0_91dc_43cc_a620_ad73779e5172.slice/crio-b63e7e0c8a66a10b8b97d5d2a4c11df88de6bf21c6a871814ba41e971d6c5be8 WatchSource:0}: Error finding container b63e7e0c8a66a10b8b97d5d2a4c11df88de6bf21c6a871814ba41e971d6c5be8: Status 404 returned error can't find the container with id b63e7e0c8a66a10b8b97d5d2a4c11df88de6bf21c6a871814ba41e971d6c5be8
Jul 13 21:41:29.510247 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.510215    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:41:29.510350 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.510293    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:41:29.510350 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.510336    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:41:29.702298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.702205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" event={"ID":"517011c0-91dc-43cc-a620-ad73779e5172","Type":"ContainerStarted","Data":"bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"}
Jul 13 21:41:29.702298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.702273    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" event={"ID":"517011c0-91dc-43cc-a620-ad73779e5172","Type":"ContainerStarted","Data":"b63e7e0c8a66a10b8b97d5d2a4c11df88de6bf21c6a871814ba41e971d6c5be8"}
Jul 13 21:41:29.721719 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:29.721660    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podStartSLOduration=0.721645962 podStartE2EDuration="721.645962ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:41:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:41:29.721189046 +0000 UTC m=+1797.233938788" watchObservedRunningTime="2026-07-13 21:41:29.721645962 +0000 UTC m=+1797.234395702"
Jul 13 21:41:30.368712 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:30.368682    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:30.373435 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:30.373409    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:30.707356 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:30.707271    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:30.708159 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:30.708138    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:33.114212 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:33.114171    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:41:33.117663 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:33.117638    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:41:37.284631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:37.284597    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:41:45.708612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.708571    2564 patch_prober.go:28] interesting pod/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 13 21:41:45.709037 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.708650    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 13 21:41:45.792270 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.792239    2564 generic.go:356] "Generic (PLEG): container finished" podID="517011c0-91dc-43cc-a620-ad73779e5172" containerID="bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075" exitCode=0
Jul 13 21:41:45.792425 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.792292    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" event={"ID":"517011c0-91dc-43cc-a620-ad73779e5172","Type":"ContainerDied","Data":"bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"}
Jul 13 21:41:45.792702 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.792688    2564 scope.go:117] "RemoveContainer" containerID="bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"
Jul 13 21:41:45.793756 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.793723    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:41:45.793824 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.793807    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:41:45.793867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:45.793844    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:41:46.800150 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:46.800112    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" event={"ID":"517011c0-91dc-43cc-a620-ad73779e5172","Type":"ContainerStarted","Data":"98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115"}
Jul 13 21:41:47.368938 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:47.368903    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:47.369150 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:47.369116    2564 patch_prober.go:28] interesting pod/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 13 21:41:47.369215 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:47.369172    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 13 21:41:48.368743 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:48.368707    2564 patch_prober.go:28] interesting pod/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 13 21:41:48.369277 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:48.368764    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 13 21:41:49.369508 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:49.369467    2564 patch_prober.go:28] interesting pod/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 13 21:41:49.369913 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:49.369539    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 13 21:41:50.369600 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:50.369567    2564 patch_prober.go:28] interesting pod/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused" start-of-body=
Jul 13 21:41:50.369978 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:50.369623    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.56:15021/healthz/ready\": dial tcp 10.134.0.56:15021: connect: connection refused"
Jul 13 21:41:51.372261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:51.372231    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:51.372658 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:51.372495    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:51.373223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:51.373207    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:41:57.462643 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:41:57.462609    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:42:09.994031 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:09.993999    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"]
Jul 13 21:42:09.994500 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:09.994227    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy" containerID="cri-o://98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115" gracePeriod=30
Jul 13 21:42:15.250632 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.250603    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:42:15.406266 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406232    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-istio-token\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406287    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517011c0-91dc-43cc-a620-ad73779e5172-istio-podinfo\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406316    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-socket\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406372    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-certs\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406436 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406415    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-data\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406633 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406446    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517011c0-91dc-43cc-a620-ad73779e5172-istiod-ca-cert\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406633 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406494    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-credential-socket\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406633 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406567    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbgwv\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-kube-api-access-kbgwv\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406633 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406596    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-envoy\") pod \"517011c0-91dc-43cc-a620-ad73779e5172\" (UID: \"517011c0-91dc-43cc-a620-ad73779e5172\") "
Jul 13 21:42:15.406840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406639    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:15.406840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406732    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:15.406946 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.406875    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:15.407023 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.407005    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.407105 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.407030    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.407105 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.407044    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.407298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.407279    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/517011c0-91dc-43cc-a620-ad73779e5172-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:42:15.407493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.407470    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-data" (OuterVolumeSpecName: "istio-data") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:15.408519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.408494    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-istio-token" (OuterVolumeSpecName: "istio-token") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:42:15.408689 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.408671    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-kube-api-access-kbgwv" (OuterVolumeSpecName: "kube-api-access-kbgwv") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "kube-api-access-kbgwv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:42:15.408884 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.408858    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/517011c0-91dc-43cc-a620-ad73779e5172-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:42:15.408984 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.408902    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "517011c0-91dc-43cc-a620-ad73779e5172" (UID: "517011c0-91dc-43cc-a620-ad73779e5172"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:42:15.508230 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.508200    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.508230 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.508226    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/517011c0-91dc-43cc-a620-ad73779e5172-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.508416 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.508236    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbgwv\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-kube-api-access-kbgwv\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.508416 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.508245    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/517011c0-91dc-43cc-a620-ad73779e5172-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.508416 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.508253    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/517011c0-91dc-43cc-a620-ad73779e5172-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.508416 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.508261    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/517011c0-91dc-43cc-a620-ad73779e5172-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:42:15.968461 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.968419    2564 generic.go:356] "Generic (PLEG): container finished" podID="517011c0-91dc-43cc-a620-ad73779e5172" containerID="98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115" exitCode=0
Jul 13 21:42:15.968652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.968486    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"
Jul 13 21:42:15.968652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.968506    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" event={"ID":"517011c0-91dc-43cc-a620-ad73779e5172","Type":"ContainerDied","Data":"98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115"}
Jul 13 21:42:15.968652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.968601    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7" event={"ID":"517011c0-91dc-43cc-a620-ad73779e5172","Type":"ContainerDied","Data":"b63e7e0c8a66a10b8b97d5d2a4c11df88de6bf21c6a871814ba41e971d6c5be8"}
Jul 13 21:42:15.968652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.968629    2564 scope.go:117] "RemoveContainer" containerID="98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115"
Jul 13 21:42:15.977905 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.977887    2564 scope.go:117] "RemoveContainer" containerID="bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"
Jul 13 21:42:15.985864 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.985844    2564 scope.go:117] "RemoveContainer" containerID="98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115"
Jul 13 21:42:15.986145 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:42:15.986125    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115\": container with ID starting with 98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115 not found: ID does not exist" containerID="98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115"
Jul 13 21:42:15.986203 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.986155    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115"} err="failed to get container status \"98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115\": rpc error: code = NotFound desc = could not find container \"98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115\": container with ID starting with 98d3813c4dc284aba07a4fa48774f6e9a505a045d34f30c14d5aba0973408115 not found: ID does not exist"
Jul 13 21:42:15.986203 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.986177    2564 scope.go:117] "RemoveContainer" containerID="bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"
Jul 13 21:42:15.986447 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:42:15.986427    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075\": container with ID starting with bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075 not found: ID does not exist" containerID="bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"
Jul 13 21:42:15.986501 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.986452    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075"} err="failed to get container status \"bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075\": rpc error: code = NotFound desc = could not find container \"bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075\": container with ID starting with bcb97e72fda99e639b98fd8023cf999ea42a4c35a933b388ae9d713a7eb7c075 not found: ID does not exist"
Jul 13 21:42:15.990983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.990961    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"]
Jul 13 21:42:15.997112 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:15.997094    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2b4e0142-openshift-default-5b7fcd6d5f-226r7"]
Jul 13 21:42:17.076904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:17.076872    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="517011c0-91dc-43cc-a620-ad73779e5172" path="/var/lib/kubelet/pods/517011c0-91dc-43cc-a620-ad73779e5172/volumes"
Jul 13 21:42:29.494298 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494262    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"]
Jul 13 21:42:29.494752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494718    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy"
Jul 13 21:42:29.494752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494730    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy"
Jul 13 21:42:29.494752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494748    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy"
Jul 13 21:42:29.494752 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494753    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494770    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494776    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bf64552-0164-4631-af88-32d16a167483" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494782    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494788    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494848    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494856    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="517011c0-91dc-43cc-a620-ad73779e5172" containerName="istio-proxy"
Jul 13 21:42:29.494898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.494864    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e992e54-f16c-455a-947d-c65dbefc5ea5" containerName="istio-proxy"
Jul 13 21:42:29.499592 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.499567    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.502010 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.501988    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ffa5db90-openshift-default-dockercfg-pv4dp\""
Jul 13 21:42:29.511830 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.511806    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"]
Jul 13 21:42:29.535787 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.535755    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-podinfo\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.535891 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.535801    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-credential-socket\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.535891 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.535840    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istiod-ca-cert\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.536004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.535891    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-socket\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.536004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.535917    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-token\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.536004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.535980    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfrl4\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-kube-api-access-nfrl4\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.536004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.536003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-data\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.536259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.536031    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-envoy\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.536259 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.536077    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-certs\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637415 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637381    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-certs\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637430    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-podinfo\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637457    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-credential-socket\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637493    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istiod-ca-cert\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637514    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-socket\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637543    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-token\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637591    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfrl4\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-kube-api-access-nfrl4\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-data\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637659    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-envoy\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.637866 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637827    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-certs\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.638090 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.637999    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-socket\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.638090 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.638044    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-credential-socket\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.638196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.638160    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-data\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.638268 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.638252    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istiod-ca-cert\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.640033 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.640012    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-podinfo\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.640162 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.640032    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-envoy\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.646478 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.646453    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-token\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.646584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.646507    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfrl4\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-kube-api-access-nfrl4\") pod \"e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") " pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.810693 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.810603    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:29.955939 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.955910    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"]
Jul 13 21:42:29.957633 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:42:29.957604    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5887ca51_5064_4b3e_bd48_673d54c8e2ba.slice/crio-e61eb42078f32bcb2e4b00b94255f916f211f6ea8a341b2d1fe07af132fa7944 WatchSource:0}: Error finding container e61eb42078f32bcb2e4b00b94255f916f211f6ea8a341b2d1fe07af132fa7944: Status 404 returned error can't find the container with id e61eb42078f32bcb2e4b00b94255f916f211f6ea8a341b2d1fe07af132fa7944
Jul 13 21:42:29.960139 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.960106    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:42:29.960196 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.960172    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:42:29.960237 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:29.960200    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:42:30.045242 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:30.045205    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" event={"ID":"5887ca51-5064-4b3e-bd48-673d54c8e2ba","Type":"ContainerStarted","Data":"c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"}
Jul 13 21:42:30.045395 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:30.045262    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" event={"ID":"5887ca51-5064-4b3e-bd48-673d54c8e2ba","Type":"ContainerStarted","Data":"e61eb42078f32bcb2e4b00b94255f916f211f6ea8a341b2d1fe07af132fa7944"}
Jul 13 21:42:30.065260 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:30.065161    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" podStartSLOduration=1.065145532 podStartE2EDuration="1.065145532s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:42:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:42:30.063639268 +0000 UTC m=+1857.576389013" watchObservedRunningTime="2026-07-13 21:42:30.065145532 +0000 UTC m=+1857.577895339"
Jul 13 21:42:30.810989 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:30.810954    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:31.815710 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:31.815681    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:32.057035 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:32.056997    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:32.058005 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:32.057987    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:37.057775 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:37.057740    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:42:46.141584 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:46.141542    2564 generic.go:356] "Generic (PLEG): container finished" podID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerID="c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32" exitCode=0
Jul 13 21:42:46.142150 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:46.141585    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" event={"ID":"5887ca51-5064-4b3e-bd48-673d54c8e2ba","Type":"ContainerDied","Data":"c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"}
Jul 13 21:42:46.142225 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:46.142187    2564 scope.go:117] "RemoveContainer" containerID="c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"
Jul 13 21:42:46.143521 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:46.143483    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:42:46.143647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:46.143577    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:42:46.143647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:46.143623    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:42:47.057620 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:47.057582    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:47.149569 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:47.149527    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" event={"ID":"5887ca51-5064-4b3e-bd48-673d54c8e2ba","Type":"ContainerStarted","Data":"ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25"}
Jul 13 21:42:47.812812 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:47.812773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:47.813017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:47.812895    2564 patch_prober.go:28] interesting pod/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 13 21:42:47.813017 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:47.812967    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 13 21:42:48.811851 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:48.811808    2564 patch_prober.go:28] interesting pod/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 13 21:42:48.812359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:48.811870    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 13 21:42:49.812078 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:49.812025    2564 patch_prober.go:28] interesting pod/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 13 21:42:49.812545 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:49.812133    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 13 21:42:50.811962 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:50.811926    2564 patch_prober.go:28] interesting pod/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Jul 13 21:42:50.812270 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:50.812240    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Jul 13 21:42:51.753648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:51.753614    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:42:51.815439 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:51.815406    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:51.815889 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:51.815803    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:42:51.816353 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:42:51.816333    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:43:11.859282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:11.859204    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:43:14.937586 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:14.937553    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"]
Jul 13 21:43:14.941254 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:14.941233    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:14.943977 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:14.943948    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 13 21:43:14.944103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:14.943994    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-fhllj\""
Jul 13 21:43:14.950844 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:14.950822    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"]
Jul 13 21:43:15.077145 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:15.077112    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c1a5e2c-514a-45e7-afbb-b4759627fe13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp\" (UID: \"2c1a5e2c-514a-45e7-afbb-b4759627fe13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:15.178379 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:15.178347    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c1a5e2c-514a-45e7-afbb-b4759627fe13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp\" (UID: \"2c1a5e2c-514a-45e7-afbb-b4759627fe13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:15.178936 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:15.178915    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c1a5e2c-514a-45e7-afbb-b4759627fe13-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp\" (UID: \"2c1a5e2c-514a-45e7-afbb-b4759627fe13\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:15.253199 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:15.253129    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:15.391434 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:15.391405    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"]
Jul 13 21:43:15.392857 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:43:15.392830    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c1a5e2c_514a_45e7_afbb_b4759627fe13.slice/crio-d5bd51ff10b42a371890a409357ef0696d688457a47cbf49a2540026ec1e13e8 WatchSource:0}: Error finding container d5bd51ff10b42a371890a409357ef0696d688457a47cbf49a2540026ec1e13e8: Status 404 returned error can't find the container with id d5bd51ff10b42a371890a409357ef0696d688457a47cbf49a2540026ec1e13e8
Jul 13 21:43:16.305830 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:16.305795    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp" event={"ID":"2c1a5e2c-514a-45e7-afbb-b4759627fe13","Type":"ContainerStarted","Data":"c7b44dea1bb8441fe963b7ada2f474c178275ff6d614872054d2daac4e32b7bb"}
Jul 13 21:43:16.306206 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:16.305853    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp" event={"ID":"2c1a5e2c-514a-45e7-afbb-b4759627fe13","Type":"ContainerStarted","Data":"d5bd51ff10b42a371890a409357ef0696d688457a47cbf49a2540026ec1e13e8"}
Jul 13 21:43:17.312886 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:17.312851    2564 generic.go:356] "Generic (PLEG): container finished" podID="2c1a5e2c-514a-45e7-afbb-b4759627fe13" containerID="c7b44dea1bb8441fe963b7ada2f474c178275ff6d614872054d2daac4e32b7bb" exitCode=0
Jul 13 21:43:17.313370 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:17.312931    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp" event={"ID":"2c1a5e2c-514a-45e7-afbb-b4759627fe13","Type":"ContainerDied","Data":"c7b44dea1bb8441fe963b7ada2f474c178275ff6d614872054d2daac4e32b7bb"}
Jul 13 21:43:17.457351 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:17.457326    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:17.602625 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:17.602596    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c1a5e2c-514a-45e7-afbb-b4759627fe13-maas-api-service-ca\") pod \"2c1a5e2c-514a-45e7-afbb-b4759627fe13\" (UID: \"2c1a5e2c-514a-45e7-afbb-b4759627fe13\") "
Jul 13 21:43:17.602966 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:17.602946    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c1a5e2c-514a-45e7-afbb-b4759627fe13-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2c1a5e2c-514a-45e7-afbb-b4759627fe13" (UID: "2c1a5e2c-514a-45e7-afbb-b4759627fe13"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:43:17.704032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:17.704000    2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c1a5e2c-514a-45e7-afbb-b4759627fe13-maas-api-service-ca\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:43:18.319255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:18.319224    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp"
Jul 13 21:43:18.319255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:18.319246    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp" event={"ID":"2c1a5e2c-514a-45e7-afbb-b4759627fe13","Type":"ContainerDied","Data":"d5bd51ff10b42a371890a409357ef0696d688457a47cbf49a2540026ec1e13e8"}
Jul 13 21:43:18.319693 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:18.319289    2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5bd51ff10b42a371890a409357ef0696d688457a47cbf49a2540026ec1e13e8"
Jul 13 21:43:29.048747 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:43:29.048710    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:44:22.358625 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:22.358594    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"]
Jul 13 21:44:22.359189 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:22.358782    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy" containerID="cri-o://ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25" gracePeriod=30
Jul 13 21:44:25.858652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:25.858569    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:44:27.621170 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.621116    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:44:27.660392 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660367    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-certs\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660535 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660413    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-socket\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660535 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660443    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-token\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660535 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660483    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istiod-ca-cert\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660535 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660512    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-podinfo\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660677    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:44:27.660748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660699    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nfrl4\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-kube-api-access-nfrl4\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660740    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-envoy\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660768    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-data\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.660904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660771    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:44:27.660904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.660796    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-credential-socket\") pod \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\" (UID: \"5887ca51-5064-4b3e-bd48-673d54c8e2ba\") "
Jul 13 21:44:27.661148 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.661120    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:44:27.661310 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.661280    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:44:27.661430 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.661310    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.661430 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.661329    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.661430 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.661341    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.661705 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.661648    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-data" (OuterVolumeSpecName: "istio-data") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:44:27.663544 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.663515    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-kube-api-access-nfrl4" (OuterVolumeSpecName: "kube-api-access-nfrl4") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "kube-api-access-nfrl4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:44:27.663648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.663569    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-token" (OuterVolumeSpecName: "istio-token") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:44:27.663648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.663617    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:44:27.663802 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.663782    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5887ca51-5064-4b3e-bd48-673d54c8e2ba" (UID: "5887ca51-5064-4b3e-bd48-673d54c8e2ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:44:27.700118 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.700095    2564 generic.go:356] "Generic (PLEG): container finished" podID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerID="ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25" exitCode=0
Jul 13 21:44:27.700213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.700143    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"
Jul 13 21:44:27.700213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.700144    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" event={"ID":"5887ca51-5064-4b3e-bd48-673d54c8e2ba","Type":"ContainerDied","Data":"ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25"}
Jul 13 21:44:27.700289 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.700243    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz" event={"ID":"5887ca51-5064-4b3e-bd48-673d54c8e2ba","Type":"ContainerDied","Data":"e61eb42078f32bcb2e4b00b94255f916f211f6ea8a341b2d1fe07af132fa7944"}
Jul 13 21:44:27.700289 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.700263    2564 scope.go:117] "RemoveContainer" containerID="ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25"
Jul 13 21:44:27.709328 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.709304    2564 scope.go:117] "RemoveContainer" containerID="c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"
Jul 13 21:44:27.717412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.717396    2564 scope.go:117] "RemoveContainer" containerID="ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25"
Jul 13 21:44:27.717661 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:44:27.717640    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25\": container with ID starting with ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25 not found: ID does not exist" containerID="ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25"
Jul 13 21:44:27.717735 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.717671    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25"} err="failed to get container status \"ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25\": rpc error: code = NotFound desc = could not find container \"ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25\": container with ID starting with ddc03ca3b17c1e874ce0a60579a1ba85fa285b0062418b4a41808a6f6aa3cd25 not found: ID does not exist"
Jul 13 21:44:27.717735 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.717694    2564 scope.go:117] "RemoveContainer" containerID="c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"
Jul 13 21:44:27.717946 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:44:27.717929    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32\": container with ID starting with c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32 not found: ID does not exist" containerID="c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"
Jul 13 21:44:27.718002 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.717954    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32"} err="failed to get container status \"c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32\": rpc error: code = NotFound desc = could not find container \"c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32\": container with ID starting with c219271e57f43e31ada5e1197bffb4aca900dd4465a911053646385ac024df32 not found: ID does not exist"
Jul 13 21:44:27.722337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.722318    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"]
Jul 13 21:44:27.728742 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.728722    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ffa5db90-openshift-default-6798f8b879-8bxxz"]
Jul 13 21:44:27.762104 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.762083    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.762104 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.762102    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.762222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.762113    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nfrl4\" (UniqueName: \"kubernetes.io/projected/5887ca51-5064-4b3e-bd48-673d54c8e2ba-kube-api-access-nfrl4\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.762222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.762126    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.762222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.762135    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:27.762222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:27.762143    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5887ca51-5064-4b3e-bd48-673d54c8e2ba-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:44:29.077292 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:29.077258    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" path="/var/lib/kubelet/pods/5887ca51-5064-4b3e-bd48-673d54c8e2ba/volumes"
Jul 13 21:44:37.155684 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:44:37.155650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:45:17.284338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284308    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"]
Jul 13 21:45:17.284823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284735    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c1a5e2c-514a-45e7-afbb-b4759627fe13" containerName="revoke-keys"
Jul 13 21:45:17.284823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284746    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c1a5e2c-514a-45e7-afbb-b4759627fe13" containerName="revoke-keys"
Jul 13 21:45:17.284823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284771    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy"
Jul 13 21:45:17.284823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284780    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy"
Jul 13 21:45:17.284823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284788    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy"
Jul 13 21:45:17.284823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284797    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy"
Jul 13 21:45:17.285015 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284869    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy"
Jul 13 21:45:17.285015 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284879    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5887ca51-5064-4b3e-bd48-673d54c8e2ba" containerName="istio-proxy"
Jul 13 21:45:17.285015 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.284886    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c1a5e2c-514a-45e7-afbb-b4759627fe13" containerName="revoke-keys"
Jul 13 21:45:17.288048 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.288030    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.290405 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.290382    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-749717-openshift-default-dockercfg-ns74g\""
Jul 13 21:45:17.302255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.302231    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"]
Jul 13 21:45:17.330659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-socket\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330663    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-data\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330692    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcvxr\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-kube-api-access-xcvxr\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330727    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-envoy\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-credential-socket\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330786    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-token\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330838    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-certs\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.330900 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330900    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-podinfo\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.331131 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.330946    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fb4806-6bd1-440a-88ba-c90a21556b4a-istiod-ca-cert\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432502 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432467    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-credential-socket\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432509    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-token\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432532    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-certs\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432572    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-podinfo\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432594    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fb4806-6bd1-440a-88ba-c90a21556b4a-istiod-ca-cert\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432639    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-socket\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.432701 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432665    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-data\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432703    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcvxr\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-kube-api-access-xcvxr\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-envoy\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-certs\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.432981    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-credential-socket\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.433259    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-socket\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433401 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.433376    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fb4806-6bd1-440a-88ba-c90a21556b4a-istiod-ca-cert\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.433600 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.433580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-data\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.435327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.435306    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-envoy\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.435481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.435458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-podinfo\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.440478 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.440450    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcvxr\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-kube-api-access-xcvxr\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.440600 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.440580    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-token\") pod \"e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") " pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.601826 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.601787    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:17.747686 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.747662    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"]
Jul 13 21:45:17.748722 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:45:17.748695    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fb4806_6bd1_440a_88ba_c90a21556b4a.slice/crio-225498b7e126bb09c63961c7cfd94846e0615ce55bfe25a34b08765f877622f2 WatchSource:0}: Error finding container 225498b7e126bb09c63961c7cfd94846e0615ce55bfe25a34b08765f877622f2: Status 404 returned error can't find the container with id 225498b7e126bb09c63961c7cfd94846e0615ce55bfe25a34b08765f877622f2
Jul 13 21:45:17.751186 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.751152    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:17.751283 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.751215    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:17.751283 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.751246    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:17.970141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.970024    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" event={"ID":"20fb4806-6bd1-440a-88ba-c90a21556b4a","Type":"ContainerStarted","Data":"eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"}
Jul 13 21:45:17.970141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:17.970113    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" event={"ID":"20fb4806-6bd1-440a-88ba-c90a21556b4a","Type":"ContainerStarted","Data":"225498b7e126bb09c63961c7cfd94846e0615ce55bfe25a34b08765f877622f2"}
Jul 13 21:45:18.012356 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:18.012306    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podStartSLOduration=1.012290408 podStartE2EDuration="1.012290408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:45:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:45:18.010403552 +0000 UTC m=+2025.523153293" watchObservedRunningTime="2026-07-13 21:45:18.012290408 +0000 UTC m=+2025.525040148"
Jul 13 21:45:18.602879 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:18.602840    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:18.607871 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:18.607841    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:18.975922 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:18.975835    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:18.976806 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:18.976786    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:25.136745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.136715    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"]
Jul 13 21:45:25.140682 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.140661    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.142980 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.142959    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-f491fd-openshift-default-dockercfg-h5gtt\""
Jul 13 21:45:25.163103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.163042    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"]
Jul 13 21:45:25.208376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208346    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-socket\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208381    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-credential-socket\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208433    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wl6xz\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-kube-api-access-wl6xz\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208502    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-certs\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208634 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553a5f69-6187-4937-98fd-ab734e89e0b8-istiod-ca-cert\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208634 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208561    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-podinfo\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208657    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-data\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208699    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-envoy\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.208805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.208728    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-token\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.309919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.309883    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-certs\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.309919 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.309930    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553a5f69-6187-4937-98fd-ab734e89e0b8-istiod-ca-cert\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.309959    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-podinfo\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310018    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-data\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310045    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-envoy\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310097    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-token\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310203    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-socket\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310446 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310227    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-credential-socket\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310498 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310280    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wl6xz\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-kube-api-access-wl6xz\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310498 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310356    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-certs\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310498 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310458    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-data\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310764    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553a5f69-6187-4937-98fd-ab734e89e0b8-istiod-ca-cert\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.310971 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310944    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-credential-socket\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.311115 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.310973    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-socket\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.312507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.312487    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-envoy\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.312838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.312816    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-podinfo\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.318006 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.317984    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-token\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.318133 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.318117    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wl6xz\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-kube-api-access-wl6xz\") pod \"e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") " pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.457986 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.457906    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:25.603767 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.603739    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"]
Jul 13 21:45:25.607755 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:45:25.607719    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod553a5f69_6187_4937_98fd_ab734e89e0b8.slice/crio-46bf01a209fa4c690d6073cb14bf8e08057951e22cab03ee9a75ffdd67182022 WatchSource:0}: Error finding container 46bf01a209fa4c690d6073cb14bf8e08057951e22cab03ee9a75ffdd67182022: Status 404 returned error can't find the container with id 46bf01a209fa4c690d6073cb14bf8e08057951e22cab03ee9a75ffdd67182022
Jul 13 21:45:25.609842 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.609806    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:25.609932 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.609889    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:25.609972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:25.609931    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:26.017073 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:26.017024    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" event={"ID":"553a5f69-6187-4937-98fd-ab734e89e0b8","Type":"ContainerStarted","Data":"4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"}
Jul 13 21:45:26.017267 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:26.017096    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" event={"ID":"553a5f69-6187-4937-98fd-ab734e89e0b8","Type":"ContainerStarted","Data":"46bf01a209fa4c690d6073cb14bf8e08057951e22cab03ee9a75ffdd67182022"}
Jul 13 21:45:26.038536 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:26.038479    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podStartSLOduration=1.038460345 podStartE2EDuration="1.038460345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:45:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:45:26.037572219 +0000 UTC m=+2033.550321956" watchObservedRunningTime="2026-07-13 21:45:26.038460345 +0000 UTC m=+2033.551210085"
Jul 13 21:45:26.458947 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:26.458912    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:26.463332 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:26.463312    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:27.022603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:27.022571    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:27.023462 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:27.023444    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:27.551737 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:27.551707    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:45:35.017230 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:35.017187    2564 patch_prober.go:28] interesting pod/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 13 21:45:35.017613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:35.017279    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 13 21:45:36.075894 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:36.075863    2564 generic.go:356] "Generic (PLEG): container finished" podID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerID="eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa" exitCode=0
Jul 13 21:45:36.076295 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:36.075930    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" event={"ID":"20fb4806-6bd1-440a-88ba-c90a21556b4a","Type":"ContainerDied","Data":"eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"}
Jul 13 21:45:36.076335 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:36.076307    2564 scope.go:117] "RemoveContainer" containerID="eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"
Jul 13 21:45:36.077291 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:36.077261    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:36.077389 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:36.077326    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:36.077389 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:36.077361    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:37.082624 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:37.082584    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" event={"ID":"20fb4806-6bd1-440a-88ba-c90a21556b4a","Type":"ContainerStarted","Data":"c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7"}
Jul 13 21:45:37.602340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:37.602303    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:37.602549 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:37.602424    2564 patch_prober.go:28] interesting pod/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 13 21:45:37.602549 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:37.602468    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 13 21:45:38.602647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:38.602207    2564 patch_prober.go:28] interesting pod/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 13 21:45:38.602647 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:38.602266    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 13 21:45:39.602498 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:39.602452    2564 patch_prober.go:28] interesting pod/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 13 21:45:39.602683 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:39.602545    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 13 21:45:40.602624 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:40.602586    2564 patch_prober.go:28] interesting pod/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Jul 13 21:45:40.603297 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:40.602892    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Jul 13 21:45:41.606800 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:41.606773    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:41.607332 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:41.607306    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:41.608015 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:41.607996    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:45:42.754844 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:42.754803    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:45:43.063311 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:43.063223    2564 patch_prober.go:28] interesting pod/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 13 21:45:43.063311 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:43.063294    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 13 21:45:44.124363 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:44.124301    2564 generic.go:356] "Generic (PLEG): container finished" podID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerID="4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e" exitCode=0
Jul 13 21:45:44.124808 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:44.124378    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" event={"ID":"553a5f69-6187-4937-98fd-ab734e89e0b8","Type":"ContainerDied","Data":"4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"}
Jul 13 21:45:44.124880 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:44.124846    2564 scope.go:117] "RemoveContainer" containerID="4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"
Jul 13 21:45:44.125836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:44.125803    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:44.125927 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:44.125895    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:44.125974 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:44.125927    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:45:45.132231 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:45.132200    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" event={"ID":"553a5f69-6187-4937-98fd-ab734e89e0b8","Type":"ContainerStarted","Data":"cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f"}
Jul 13 21:45:45.458630 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:45.458540    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:45.458783 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:45.458757    2564 patch_prober.go:28] interesting pod/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 13 21:45:45.458826 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:45.458797    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 13 21:45:46.458270 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:46.458234    2564 patch_prober.go:28] interesting pod/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 13 21:45:46.458690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:46.458299    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 13 21:45:47.458533 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.458475    2564 patch_prober.go:28] interesting pod/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 13 21:45:47.459114 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.458591    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 13 21:45:47.472762 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.472733    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"]
Jul 13 21:45:47.477167 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.477144    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.479797 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.479771    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-749717\"/\"kube-root-ca.crt\""
Jul 13 21:45:47.480606 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.480257    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-749717\"/\"test-model-749717-kserve-self-signed-certs\""
Jul 13 21:45:47.480606 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.480281    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-749717\"/\"openshift-service-ca.crt\""
Jul 13 21:45:47.480606 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.480501    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-749717\"/\"default-dockercfg-vp2t8\""
Jul 13 21:45:47.487308 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.487284    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"]
Jul 13 21:45:47.560838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.560668    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-dshm\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.560838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.560706    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tls-certs\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.560838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.560732    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmv85\" (UniqueName: \"kubernetes.io/projected/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-kube-api-access-jmv85\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.560838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.560778    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tmp-dir\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.560838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.560809    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-home\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.561175 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.560858    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-model-cache\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.662275 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.662235    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-home\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.662617 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.662598    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-model-cache\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.662816 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.662682    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-home\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.662976 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.662946    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-model-cache\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.663085 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.662958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-dshm\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.663208 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.663194    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tls-certs\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.663331 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.663310    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmv85\" (UniqueName: \"kubernetes.io/projected/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-kube-api-access-jmv85\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.663476 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.663410    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tmp-dir\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.663784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.663758    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tmp-dir\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.665873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.665855    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-dshm\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.666433 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.666415    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tls-certs\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.670997 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.670965    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmv85\" (UniqueName: \"kubernetes.io/projected/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-kube-api-access-jmv85\") pod \"test-model-749717-kserve-6774cc4b8c-fcfxd\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") " pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.790892 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.790793    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:47.977477 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:47.977449    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"]
Jul 13 21:45:47.978881 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:45:47.978848    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9df39e79_52f4_4c5b_aae6_90811fd7c6ad.slice/crio-0a070693e8c17ef80d35f4870b7b1a109ac8b0faa25877e8f21c1f79dcf738dd WatchSource:0}: Error finding container 0a070693e8c17ef80d35f4870b7b1a109ac8b0faa25877e8f21c1f79dcf738dd: Status 404 returned error can't find the container with id 0a070693e8c17ef80d35f4870b7b1a109ac8b0faa25877e8f21c1f79dcf738dd
Jul 13 21:45:48.150137 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:48.150102    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd" event={"ID":"9df39e79-52f4-4c5b-aae6-90811fd7c6ad","Type":"ContainerStarted","Data":"929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591"}
Jul 13 21:45:48.150289 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:48.150148    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd" event={"ID":"9df39e79-52f4-4c5b-aae6-90811fd7c6ad","Type":"ContainerStarted","Data":"0a070693e8c17ef80d35f4870b7b1a109ac8b0faa25877e8f21c1f79dcf738dd"}
Jul 13 21:45:48.167178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:48.167127    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd" podStartSLOduration=1.167111476 podStartE2EDuration="1.167111476s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:45:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:45:48.165906021 +0000 UTC m=+2055.678655762" watchObservedRunningTime="2026-07-13 21:45:48.167111476 +0000 UTC m=+2055.679861217"
Jul 13 21:45:48.458860 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:48.458825    2564 patch_prober.go:28] interesting pod/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused" start-of-body=
Jul 13 21:45:48.459282 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:48.458879    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.60:15021/healthz/ready\": dial tcp 10.134.0.60:15021: connect: connection refused"
Jul 13 21:45:49.462218 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:49.462184    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:49.462659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:49.462466    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:49.463154 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:49.463137    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:45:53.361255 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:53.361220    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:45:57.791570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:57.791531    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:57.792140 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:57.791581    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:57.804716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:57.804686    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:45:58.214826 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:45:58.214791    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:46:16.364361 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:16.364330    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:46:30.152233 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:30.152201    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:46:30.937512 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:30.937473    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"]
Jul 13 21:46:33.150347 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:33.150322    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:46:33.155244 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:33.155225    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:46:42.659369 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:42.659337    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:46:45.958622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:45.958569    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd" podUID="9df39e79-52f4-4c5b-aae6-90811fd7c6ad" containerName="main" containerID="cri-o://929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591" gracePeriod=45
Jul 13 21:46:46.217748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.217689    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:46:46.356729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356696    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-model-cache\") pod \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") "
Jul 13 21:46:46.356729 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356732    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-dshm\") pod \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") "
Jul 13 21:46:46.356982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356759    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmv85\" (UniqueName: \"kubernetes.io/projected/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-kube-api-access-jmv85\") pod \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") "
Jul 13 21:46:46.356982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356781    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-home\") pod \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") "
Jul 13 21:46:46.356982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356819    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tmp-dir\") pod \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") "
Jul 13 21:46:46.356982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356901    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tls-certs\") pod \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\" (UID: \"9df39e79-52f4-4c5b-aae6-90811fd7c6ad\") "
Jul 13 21:46:46.357238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.356990    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-model-cache" (OuterVolumeSpecName: "model-cache") pod "9df39e79-52f4-4c5b-aae6-90811fd7c6ad" (UID: "9df39e79-52f4-4c5b-aae6-90811fd7c6ad"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:46.357238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.357093    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-home" (OuterVolumeSpecName: "home") pod "9df39e79-52f4-4c5b-aae6-90811fd7c6ad" (UID: "9df39e79-52f4-4c5b-aae6-90811fd7c6ad"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:46.357238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.357168    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9df39e79-52f4-4c5b-aae6-90811fd7c6ad" (UID: "9df39e79-52f4-4c5b-aae6-90811fd7c6ad"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:46.357396 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.357258    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-model-cache\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:46:46.357396 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.357273    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-home\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:46:46.357396 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.357304    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tmp-dir\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:46:46.359011 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.358983    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9df39e79-52f4-4c5b-aae6-90811fd7c6ad" (UID: "9df39e79-52f4-4c5b-aae6-90811fd7c6ad"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:46:46.359141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.359008    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-kube-api-access-jmv85" (OuterVolumeSpecName: "kube-api-access-jmv85") pod "9df39e79-52f4-4c5b-aae6-90811fd7c6ad" (UID: "9df39e79-52f4-4c5b-aae6-90811fd7c6ad"). InnerVolumeSpecName "kube-api-access-jmv85". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:46:46.359141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.358987    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-dshm" (OuterVolumeSpecName: "dshm") pod "9df39e79-52f4-4c5b-aae6-90811fd7c6ad" (UID: "9df39e79-52f4-4c5b-aae6-90811fd7c6ad"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:46:46.458061 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.458031    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-tls-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:46:46.458217 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.458083    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-dshm\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:46:46.458217 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.458093    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmv85\" (UniqueName: \"kubernetes.io/projected/9df39e79-52f4-4c5b-aae6-90811fd7c6ad-kube-api-access-jmv85\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:46:46.472650 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.472579    2564 generic.go:356] "Generic (PLEG): container finished" podID="9df39e79-52f4-4c5b-aae6-90811fd7c6ad" containerID="929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591" exitCode=0
Jul 13 21:46:46.472650 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.472632    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"
Jul 13 21:46:46.472801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.472668    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd" event={"ID":"9df39e79-52f4-4c5b-aae6-90811fd7c6ad","Type":"ContainerDied","Data":"929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591"}
Jul 13 21:46:46.472801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.472708    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd" event={"ID":"9df39e79-52f4-4c5b-aae6-90811fd7c6ad","Type":"ContainerDied","Data":"0a070693e8c17ef80d35f4870b7b1a109ac8b0faa25877e8f21c1f79dcf738dd"}
Jul 13 21:46:46.472801 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.472725    2564 scope.go:117] "RemoveContainer" containerID="929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591"
Jul 13 21:46:46.482035 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.482018    2564 scope.go:117] "RemoveContainer" containerID="929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591"
Jul 13 21:46:46.482289 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:46:46.482271    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591\": container with ID starting with 929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591 not found: ID does not exist" containerID="929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591"
Jul 13 21:46:46.482340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.482297    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591"} err="failed to get container status \"929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591\": rpc error: code = NotFound desc = could not find container \"929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591\": container with ID starting with 929444622ae9c010e81db5ea681c80fa50b201d3d5e585e7893edce72344d591 not found: ID does not exist"
Jul 13 21:46:46.493461 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.493439    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"]
Jul 13 21:46:46.498383 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:46.498356    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-749717/test-model-749717-kserve-6774cc4b8c-fcfxd"]
Jul 13 21:46:47.077287 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:47.077255    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9df39e79-52f4-4c5b-aae6-90811fd7c6ad" path="/var/lib/kubelet/pods/9df39e79-52f4-4c5b-aae6-90811fd7c6ad/volumes"
Jul 13 21:46:52.954175 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:52.954129    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:46:57.874042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:57.874010    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"]
Jul 13 21:46:57.874519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:46:57.874225    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy" containerID="cri-o://c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7" gracePeriod=30
Jul 13 21:47:03.131742 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.131715    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:47:03.207152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207048    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-data\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207126    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-token\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207144    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-envoy\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207186    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-podinfo\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207213    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-credential-socket\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207242    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-socket\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207286    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcvxr\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-kube-api-access-xcvxr\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207315    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-certs\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207497    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:03.207679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207580    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:03.207794 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207750    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fb4806-6bd1-440a-88ba-c90a21556b4a-istiod-ca-cert\") pod \"20fb4806-6bd1-440a-88ba-c90a21556b4a\" (UID: \"20fb4806-6bd1-440a-88ba-c90a21556b4a\") "
Jul 13 21:47:03.207794 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.207774    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:03.208093 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208068    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20fb4806-6bd1-440a-88ba-c90a21556b4a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:47:03.208093 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208045    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-data" (OuterVolumeSpecName: "istio-data") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:03.208325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208189    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.208325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208201    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.208325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208211    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.208325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208219    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20fb4806-6bd1-440a-88ba-c90a21556b4a-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.208325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.208228    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.209394 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.209366    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:47:03.209575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.209546    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-kube-api-access-xcvxr" (OuterVolumeSpecName: "kube-api-access-xcvxr") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "kube-api-access-xcvxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:47:03.209773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.209752    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-token" (OuterVolumeSpecName: "istio-token") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:47:03.209848 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.209772    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20fb4806-6bd1-440a-88ba-c90a21556b4a" (UID: "20fb4806-6bd1-440a-88ba-c90a21556b4a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:03.309745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.309710    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.309745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.309740    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.309745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.309748    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20fb4806-6bd1-440a-88ba-c90a21556b4a-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.309982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.309757    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcvxr\" (UniqueName: \"kubernetes.io/projected/20fb4806-6bd1-440a-88ba-c90a21556b4a-kube-api-access-xcvxr\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:03.570434 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.570403    2564 generic.go:356] "Generic (PLEG): container finished" podID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerID="c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7" exitCode=0
Jul 13 21:47:03.570603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.570449    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"
Jul 13 21:47:03.570603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.570487    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" event={"ID":"20fb4806-6bd1-440a-88ba-c90a21556b4a","Type":"ContainerDied","Data":"c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7"}
Jul 13 21:47:03.570603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.570540    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h" event={"ID":"20fb4806-6bd1-440a-88ba-c90a21556b4a","Type":"ContainerDied","Data":"225498b7e126bb09c63961c7cfd94846e0615ce55bfe25a34b08765f877622f2"}
Jul 13 21:47:03.570603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.570562    2564 scope.go:117] "RemoveContainer" containerID="c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7"
Jul 13 21:47:03.581897 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.581885    2564 scope.go:117] "RemoveContainer" containerID="eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"
Jul 13 21:47:03.592152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.592126    2564 scope.go:117] "RemoveContainer" containerID="c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7"
Jul 13 21:47:03.592494 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:47:03.592437    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7\": container with ID starting with c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7 not found: ID does not exist" containerID="c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7"
Jul 13 21:47:03.592494 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.592470    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7"} err="failed to get container status \"c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7\": rpc error: code = NotFound desc = could not find container \"c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7\": container with ID starting with c63a1987b7de33d09339daea6ec63bf104ec4ce7db7dc793564df852c3b2b5c7 not found: ID does not exist"
Jul 13 21:47:03.592494 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.592494    2564 scope.go:117] "RemoveContainer" containerID="eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"
Jul 13 21:47:03.592772 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:47:03.592746    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa\": container with ID starting with eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa not found: ID does not exist" containerID="eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"
Jul 13 21:47:03.592877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.592776    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa"} err="failed to get container status \"eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa\": rpc error: code = NotFound desc = could not find container \"eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa\": container with ID starting with eed7542113b45bc77dec10b4818f6f27187d7c40dbf7bcec1359cbebc88bc7fa not found: ID does not exist"
Jul 13 21:47:03.594379 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.594359    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"]
Jul 13 21:47:03.598173 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:03.598156    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-749717-openshift-default-6fff7bd59f-blf9h"]
Jul 13 21:47:05.082904 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:05.082868    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" path="/var/lib/kubelet/pods/20fb4806-6bd1-440a-88ba-c90a21556b4a/volumes"
Jul 13 21:47:12.056336 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:12.056303    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:47:23.996429 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:23.996350    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"]
Jul 13 21:47:23.996877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:23.996595    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy" containerID="cri-o://cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f" gracePeriod=30
Jul 13 21:47:24.902359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902324    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"]
Jul 13 21:47:24.902778 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902765    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy"
Jul 13 21:47:24.902843 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902779    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy"
Jul 13 21:47:24.902843 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902803    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9df39e79-52f4-4c5b-aae6-90811fd7c6ad" containerName="main"
Jul 13 21:47:24.902843 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902808    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9df39e79-52f4-4c5b-aae6-90811fd7c6ad" containerName="main"
Jul 13 21:47:24.902999 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902875    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy"
Jul 13 21:47:24.902999 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902884    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9df39e79-52f4-4c5b-aae6-90811fd7c6ad" containerName="main"
Jul 13 21:47:24.902999 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.902891    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy"
Jul 13 21:47:24.906193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.906169    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:24.910782 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.910762    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-459a1d-openshift-default-dockercfg-ph57p\""
Jul 13 21:47:24.919325 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:24.919299    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"]
Jul 13 21:47:25.021803 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.021765    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-socket\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.021811    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-data\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.021844    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-credential-socket\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.021864    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnqf5\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-kube-api-access-cnqf5\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.021894    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b036c072-b919-4970-80f3-8216e0996495-istio-podinfo\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.021985    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b036c072-b919-4970-80f3-8216e0996495-istiod-ca-cert\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.022025    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-envoy\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.022048    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-istio-token\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.022213 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.022081    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-certs\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123329 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123297    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b036c072-b919-4970-80f3-8216e0996495-istio-podinfo\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123365    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b036c072-b919-4970-80f3-8216e0996495-istiod-ca-cert\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123395    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-envoy\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123415    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-istio-token\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123437    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-certs\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123515 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123484    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-socket\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123528    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-data\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123570    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-credential-socket\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.123779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.123597    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnqf5\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-kube-api-access-cnqf5\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.124080 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.124033    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-data\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.124208 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.124119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-certs\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.124208 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.124144    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-credential-socket\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.124208 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.124185    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-socket\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.124340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.124305    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b036c072-b919-4970-80f3-8216e0996495-istiod-ca-cert\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.126184 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.126167    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-envoy\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.126343 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.126326    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b036c072-b919-4970-80f3-8216e0996495-istio-podinfo\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.131048 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.131028    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-istio-token\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.131180 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.131126    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnqf5\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-kube-api-access-cnqf5\") pod \"e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") " pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.217323 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.217247    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:25.365507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.365481    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"]
Jul 13 21:47:25.366897 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:47:25.366867    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb036c072_b919_4970_80f3_8216e0996495.slice/crio-ce38c10be4c167d9f8f0d4e4ff82817120f474bb448c119bb12a35040f379b03 WatchSource:0}: Error finding container ce38c10be4c167d9f8f0d4e4ff82817120f474bb448c119bb12a35040f379b03: Status 404 returned error can't find the container with id ce38c10be4c167d9f8f0d4e4ff82817120f474bb448c119bb12a35040f379b03
Jul 13 21:47:25.369123 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.369091    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:25.369195 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.369157    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:25.369195 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.369184    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:25.703658 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.703619    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" event={"ID":"b036c072-b919-4970-80f3-8216e0996495","Type":"ContainerStarted","Data":"14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"}
Jul 13 21:47:25.703849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.703680    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" event={"ID":"b036c072-b919-4970-80f3-8216e0996495","Type":"ContainerStarted","Data":"ce38c10be4c167d9f8f0d4e4ff82817120f474bb448c119bb12a35040f379b03"}
Jul 13 21:47:25.723379 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:25.723328    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podStartSLOduration=1.7233093830000001 podStartE2EDuration="1.723309383s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:47:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:47:25.722594834 +0000 UTC m=+2153.235344576" watchObservedRunningTime="2026-07-13 21:47:25.723309383 +0000 UTC m=+2153.236059125"
Jul 13 21:47:26.218008 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:26.217974    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:26.222906 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:26.222882    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:26.709007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:26.708980    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:26.710018 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:26.709999    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:29.266355 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.266297    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:47:29.367456 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367427    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553a5f69-6187-4937-98fd-ab734e89e0b8-istiod-ca-cert\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367476    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-certs\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367546    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-socket\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367577    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-podinfo\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367616 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367605    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-data\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367633    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-envoy\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367678    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wl6xz\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-kube-api-access-wl6xz\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367707    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-credential-socket\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-token\") pod \"553a5f69-6187-4937-98fd-ab734e89e0b8\" (UID: \"553a5f69-6187-4937-98fd-ab734e89e0b8\") "
Jul 13 21:47:29.367916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367834    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/553a5f69-6187-4937-98fd-ab734e89e0b8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:47:29.367916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.367880    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:29.368257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.368044    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:29.368257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.368214    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.368257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.368230    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.368257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.368241    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/553a5f69-6187-4937-98fd-ab734e89e0b8-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.368466 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.368374    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:29.368559 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.368535    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-data" (OuterVolumeSpecName: "istio-data") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:29.369928 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.369905    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-token" (OuterVolumeSpecName: "istio-token") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:47:29.370089 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.370047    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:47:29.370272 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.370248    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:47:29.370272 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.370264    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-kube-api-access-wl6xz" (OuterVolumeSpecName: "kube-api-access-wl6xz") pod "553a5f69-6187-4937-98fd-ab734e89e0b8" (UID: "553a5f69-6187-4937-98fd-ab734e89e0b8"). InnerVolumeSpecName "kube-api-access-wl6xz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:47:29.468733 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.468704    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.468733 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.468729    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.468733 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.468739    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wl6xz\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-kube-api-access-wl6xz\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.468934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.468747    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.468934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.468757    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/553a5f69-6187-4937-98fd-ab734e89e0b8-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.468934 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.468765    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/553a5f69-6187-4937-98fd-ab734e89e0b8-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:47:29.726264 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.726230    2564 generic.go:356] "Generic (PLEG): container finished" podID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerID="cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f" exitCode=0
Jul 13 21:47:29.726438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.726293    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"
Jul 13 21:47:29.726438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.726295    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" event={"ID":"553a5f69-6187-4937-98fd-ab734e89e0b8","Type":"ContainerDied","Data":"cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f"}
Jul 13 21:47:29.726438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.726377    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw" event={"ID":"553a5f69-6187-4937-98fd-ab734e89e0b8","Type":"ContainerDied","Data":"46bf01a209fa4c690d6073cb14bf8e08057951e22cab03ee9a75ffdd67182022"}
Jul 13 21:47:29.726438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.726400    2564 scope.go:117] "RemoveContainer" containerID="cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f"
Jul 13 21:47:29.735972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.735955    2564 scope.go:117] "RemoveContainer" containerID="4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"
Jul 13 21:47:29.743737 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.743722    2564 scope.go:117] "RemoveContainer" containerID="cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f"
Jul 13 21:47:29.743977 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:47:29.743958    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f\": container with ID starting with cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f not found: ID does not exist" containerID="cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f"
Jul 13 21:47:29.744032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.743986    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f"} err="failed to get container status \"cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f\": rpc error: code = NotFound desc = could not find container \"cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f\": container with ID starting with cb9720ff274843bc5f9921c48df38ec062965027e3de32e26841fc52adb4bc3f not found: ID does not exist"
Jul 13 21:47:29.744032 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.744003    2564 scope.go:117] "RemoveContainer" containerID="4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"
Jul 13 21:47:29.744277 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:47:29.744261    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e\": container with ID starting with 4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e not found: ID does not exist" containerID="4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"
Jul 13 21:47:29.744331 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.744284    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e"} err="failed to get container status \"4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e\": rpc error: code = NotFound desc = could not find container \"4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e\": container with ID starting with 4fa32fdb6334461b6f5c07406c17790ea4c626a948a84e29c5036d4797c1e65e not found: ID does not exist"
Jul 13 21:47:29.747874 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.747853    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"]
Jul 13 21:47:29.751955 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:29.751934    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-f491fd-openshift-default-67884cd547-xcncw"]
Jul 13 21:47:31.077464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:31.077431    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" path="/var/lib/kubelet/pods/553a5f69-6187-4937-98fd-ab734e89e0b8/volumes"
Jul 13 21:47:32.881189 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881154    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"]
Jul 13 21:47:32.881643 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881628    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy"
Jul 13 21:47:32.881688 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881644    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy"
Jul 13 21:47:32.881688 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881670    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy"
Jul 13 21:47:32.881688 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881675    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fb4806-6bd1-440a-88ba-c90a21556b4a" containerName="istio-proxy"
Jul 13 21:47:32.881786 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881748    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy"
Jul 13 21:47:32.881786 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.881757    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy"
Jul 13 21:47:32.886529 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.886508    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.889758 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.889735    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7b2acc-openshift-default-dockercfg-x8drq\""
Jul 13 21:47:32.892705 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892686    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-podinfo\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.892792 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892734    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-token\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.892792 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892783    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istiod-ca-cert\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.892878 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892855    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-socket\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.892878 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892873    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxr6g\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-kube-api-access-qxr6g\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.892987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892929    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-data\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.892987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.892968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-credential-socket\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.893128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.893014    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-envoy\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.893128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.893068    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-certs\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.900176 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.899828    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"]
Jul 13 21:47:32.993769 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993724    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-envoy\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.993769 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-certs\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993833    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-podinfo\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993874    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-token\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993894    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istiod-ca-cert\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993951    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-socket\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993966    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxr6g\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-kube-api-access-qxr6g\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.993989    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-data\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.994008    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-credential-socket\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994657 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.994631    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-credential-socket\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.994892 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.994806    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istiod-ca-cert\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.995347 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.995298    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-data\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.995456 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.995374    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-certs\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.995680 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.995660    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-socket\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.997171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.996685    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-envoy\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:32.997171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:32.997120    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-podinfo\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:33.004226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.004200    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxr6g\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-kube-api-access-qxr6g\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:33.004888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.004863    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-token\") pod \"e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") " pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:33.201820 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.201728    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-7b2acc-openshift-default-dockercfg-x8drq\""
Jul 13 21:47:33.209308 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.209278    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:33.355279 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.355253    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"]
Jul 13 21:47:33.356406 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:47:33.356382    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69ec8b7_6bc2_4efe_9453_96891f5a5db9.slice/crio-c96606395ddeddbaa3bcbab6de966e522fed8357e5d47459be8110ebd9992098 WatchSource:0}: Error finding container c96606395ddeddbaa3bcbab6de966e522fed8357e5d47459be8110ebd9992098: Status 404 returned error can't find the container with id c96606395ddeddbaa3bcbab6de966e522fed8357e5d47459be8110ebd9992098
Jul 13 21:47:33.358564 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.358529    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:33.358667 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.358607    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:33.358667 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.358650    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:33.755855 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.755814    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" event={"ID":"d69ec8b7-6bc2-4efe-9453-96891f5a5db9","Type":"ContainerStarted","Data":"a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"}
Jul 13 21:47:33.756082 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.755916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" event={"ID":"d69ec8b7-6bc2-4efe-9453-96891f5a5db9","Type":"ContainerStarted","Data":"c96606395ddeddbaa3bcbab6de966e522fed8357e5d47459be8110ebd9992098"}
Jul 13 21:47:33.806840 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.806776    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" podStartSLOduration=1.806755938 podStartE2EDuration="1.806755938s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:47:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:47:33.803778982 +0000 UTC m=+2161.316528736" watchObservedRunningTime="2026-07-13 21:47:33.806755938 +0000 UTC m=+2161.319505681"
Jul 13 21:47:33.819740 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:33.819712    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:47:34.210528 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:34.210500    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:34.214873 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:34.214850    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:34.761309 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:34.761278    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:34.762276 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:34.762255    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:41.710076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.709972    2564 patch_prober.go:28] interesting pod/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 13 21:47:41.710076 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.710021    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 13 21:47:41.801499 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.801459    2564 generic.go:356] "Generic (PLEG): container finished" podID="b036c072-b919-4970-80f3-8216e0996495" containerID="14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee" exitCode=0
Jul 13 21:47:41.801653 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.801519    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" event={"ID":"b036c072-b919-4970-80f3-8216e0996495","Type":"ContainerDied","Data":"14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"}
Jul 13 21:47:41.802027 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.802010    2564 scope.go:117] "RemoveContainer" containerID="14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"
Jul 13 21:47:41.803119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.803087    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:41.803202 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.803160    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:41.803202 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:41.803191    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:42.813419 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:42.813371    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" event={"ID":"b036c072-b919-4970-80f3-8216e0996495","Type":"ContainerStarted","Data":"d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64"}
Jul 13 21:47:43.217712 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:43.217677    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:43.218071 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:43.217968    2564 patch_prober.go:28] interesting pod/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 13 21:47:43.218297 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:43.218272    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 13 21:47:44.217887 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:44.217842    2564 patch_prober.go:28] interesting pod/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 13 21:47:44.218365 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:44.217907    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 13 21:47:45.218081 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:45.218017    2564 patch_prober.go:28] interesting pod/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 13 21:47:45.218590 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:45.218097    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 13 21:47:46.218370 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:46.217879    2564 patch_prober.go:28] interesting pod/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused" start-of-body=
Jul 13 21:47:46.218370 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:46.217943    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.62:15021/healthz/ready\": dial tcp 10.134.0.62:15021: connect: connection refused"
Jul 13 21:47:47.221888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:47.221850    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:47.222333 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:47.222269    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:47.222926 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:47.222905    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:47:48.858039 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:48.858009    2564 generic.go:356] "Generic (PLEG): container finished" podID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerID="a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578" exitCode=0
Jul 13 21:47:48.858386 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:48.858083    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" event={"ID":"d69ec8b7-6bc2-4efe-9453-96891f5a5db9","Type":"ContainerDied","Data":"a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"}
Jul 13 21:47:48.858652 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:48.858637    2564 scope.go:117] "RemoveContainer" containerID="a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"
Jul 13 21:47:48.859613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:48.859582    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:48.859679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:48.859666    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:48.859730 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:48.859717    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:47:49.761805 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:49.761765    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:49.865992 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:49.865947    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" event={"ID":"d69ec8b7-6bc2-4efe-9453-96891f5a5db9","Type":"ContainerStarted","Data":"b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632"}
Jul 13 21:47:50.211739 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:50.211703    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:50.212128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:50.212103    2564 patch_prober.go:28] interesting pod/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 13 21:47:50.212324 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:50.212281    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 13 21:47:50.767886 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:50.767852    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:47:51.210714 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:51.210677    2564 patch_prober.go:28] interesting pod/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 13 21:47:51.211314 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:51.211279    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 13 21:47:52.210141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:52.210105    2564 patch_prober.go:28] interesting pod/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 13 21:47:52.210421 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:52.210389    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 13 21:47:53.209721 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:53.209677    2564 patch_prober.go:28] interesting pod/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused" start-of-body=
Jul 13 21:47:53.210243 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:53.209744    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.63:15021/healthz/ready\": dial tcp 10.134.0.63:15021: connect: connection refused"
Jul 13 21:47:54.214449 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:54.214419    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:54.214863 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:54.214679    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:47:54.215268 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:47:54.215250    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:48:03.353836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:03.353804    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:48:14.157861 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:14.157826    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"]
Jul 13 21:48:14.158417 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:14.158081    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" containerID="cri-o://d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64" gracePeriod=30
Jul 13 21:48:17.224580 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:17.224535    2564 patch_prober.go:28] interesting pod/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Jul 13 21:48:17.224979 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:17.224594    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 13 21:48:19.408793 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.408768    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:48:19.493570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493494    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnqf5\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-kube-api-access-cnqf5\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493570 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493550    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-socket\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493583    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b036c072-b919-4970-80f3-8216e0996495-istio-podinfo\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493600    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b036c072-b919-4970-80f3-8216e0996495-istiod-ca-cert\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493616    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-certs\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493652    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-data\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493707    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-envoy\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.493784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-credential-socket\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.494119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493811    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-istio-token\") pod \"b036c072-b919-4970-80f3-8216e0996495\" (UID: \"b036c072-b919-4970-80f3-8216e0996495\") "
Jul 13 21:48:19.494119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493927    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:19.494119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.493954    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:19.494119 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494045    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b036c072-b919-4970-80f3-8216e0996495-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:48:19.494338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494161    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:19.494338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494291    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.494338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494304    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.494338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494314    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b036c072-b919-4970-80f3-8216e0996495-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.494338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494322    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.494722 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.494694    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-data" (OuterVolumeSpecName: "istio-data") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:19.495898 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.495872    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b036c072-b919-4970-80f3-8216e0996495-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:48:19.496013 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.495963    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:19.496013 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.495977    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-istio-token" (OuterVolumeSpecName: "istio-token") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:48:19.496013 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.496004    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-kube-api-access-cnqf5" (OuterVolumeSpecName: "kube-api-access-cnqf5") pod "b036c072-b919-4970-80f3-8216e0996495" (UID: "b036c072-b919-4970-80f3-8216e0996495"). InnerVolumeSpecName "kube-api-access-cnqf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:48:19.595717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.595691    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b036c072-b919-4970-80f3-8216e0996495-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.595717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.595714    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.595865 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.595723    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b036c072-b919-4970-80f3-8216e0996495-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.595865 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.595731    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:19.595865 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:19.595739    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnqf5\" (UniqueName: \"kubernetes.io/projected/b036c072-b919-4970-80f3-8216e0996495-kube-api-access-cnqf5\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:20.041839 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.041802    2564 generic.go:356] "Generic (PLEG): container finished" podID="b036c072-b919-4970-80f3-8216e0996495" containerID="d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64" exitCode=0
Jul 13 21:48:20.042042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.041853    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"
Jul 13 21:48:20.042042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.041863    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" event={"ID":"b036c072-b919-4970-80f3-8216e0996495","Type":"ContainerDied","Data":"d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64"}
Jul 13 21:48:20.042042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.041905    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq" event={"ID":"b036c072-b919-4970-80f3-8216e0996495","Type":"ContainerDied","Data":"ce38c10be4c167d9f8f0d4e4ff82817120f474bb448c119bb12a35040f379b03"}
Jul 13 21:48:20.042042 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.041928    2564 scope.go:117] "RemoveContainer" containerID="d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64"
Jul 13 21:48:20.052822 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.052803    2564 scope.go:117] "RemoveContainer" containerID="14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"
Jul 13 21:48:20.060888 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.060872    2564 scope.go:117] "RemoveContainer" containerID="d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64"
Jul 13 21:48:20.061159 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:48:20.061137    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64\": container with ID starting with d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64 not found: ID does not exist" containerID="d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64"
Jul 13 21:48:20.061242 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.061171    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64"} err="failed to get container status \"d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64\": rpc error: code = NotFound desc = could not find container \"d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64\": container with ID starting with d707d09a1e67454bc032270040623f069d25331375043fa0847fe4807d77eb64 not found: ID does not exist"
Jul 13 21:48:20.061242 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.061196    2564 scope.go:117] "RemoveContainer" containerID="14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"
Jul 13 21:48:20.061458 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:48:20.061442    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee\": container with ID starting with 14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee not found: ID does not exist" containerID="14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"
Jul 13 21:48:20.061497 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.061465    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee"} err="failed to get container status \"14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee\": rpc error: code = NotFound desc = could not find container \"14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee\": container with ID starting with 14c121a7085e0af8eb16e7453d3836dd99839ccaf4801ec1343270f60f7323ee not found: ID does not exist"
Jul 13 21:48:20.064773 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.064751    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"]
Jul 13 21:48:20.069978 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:20.069956    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-459a1d-openshift-default-7fc56d558c-77jnq"]
Jul 13 21:48:21.076972 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:21.076941    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b036c072-b919-4970-80f3-8216e0996495" path="/var/lib/kubelet/pods/b036c072-b919-4970-80f3-8216e0996495/volumes"
Jul 13 21:48:28.255827 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:28.255793    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:48:40.305114 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:40.305076    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"]
Jul 13 21:48:40.305507 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:40.305289    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy" containerID="cri-o://b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632" gracePeriod=30
Jul 13 21:48:41.244479 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.244442    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"]
Jul 13 21:48:41.245131 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245107    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy"
Jul 13 21:48:41.245131 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245130    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy"
Jul 13 21:48:41.245316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245181    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy"
Jul 13 21:48:41.245316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245190    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="553a5f69-6187-4937-98fd-ab734e89e0b8" containerName="istio-proxy"
Jul 13 21:48:41.245316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245199    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy"
Jul 13 21:48:41.245316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245206    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy"
Jul 13 21:48:41.245316 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245308    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy"
Jul 13 21:48:41.245542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.245324    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="b036c072-b919-4970-80f3-8216e0996495" containerName="istio-proxy"
Jul 13 21:48:41.248346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.248324    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.251727 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.251703    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-8139bf-openshift-default-dockercfg-8kmhr\""
Jul 13 21:48:41.260935 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.260911    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"]
Jul 13 21:48:41.404520 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404486    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-socket\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404533    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pccq\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-kube-api-access-5pccq\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404584    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-data\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404626    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-podinfo\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404643    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-token\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404678    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istiod-ca-cert\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404704    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-certs\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404764    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-envoy\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.404910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.404813    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-credential-socket\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.505963 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.505877    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-envoy\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.505963 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.505935    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-credential-socket\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.505967    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-socket\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506011    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pccq\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-kube-api-access-5pccq\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506091    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-data\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506141    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-podinfo\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506165    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-token\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506231    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istiod-ca-cert\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506391    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-credential-socket\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506481 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506434    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-certs\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506642 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506603    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-data\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506707    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-socket\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.506793 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.506733    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-certs\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.507149 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.507130    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istiod-ca-cert\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.508679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.508658    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-envoy\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.508959 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.508938    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-podinfo\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.514451 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.514427    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-token\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.514889 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.514871    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pccq\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-kube-api-access-5pccq\") pod \"e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") " pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.558941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.558922    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:41.696464 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.696435    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"]
Jul 13 21:48:41.697898 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:48:41.697871    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a47b26b_4d2e_49cf_814b_424ab3bc1dfb.slice/crio-9f1994f4a013b5865d5a347588808bf846136d1c2be8fe2921cbb4ddcf429486 WatchSource:0}: Error finding container 9f1994f4a013b5865d5a347588808bf846136d1c2be8fe2921cbb4ddcf429486: Status 404 returned error can't find the container with id 9f1994f4a013b5865d5a347588808bf846136d1c2be8fe2921cbb4ddcf429486
Jul 13 21:48:41.700851 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.700812    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:41.700920 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.700904    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:41.700962 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:41.700951    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:42.162319 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:42.162284    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" event={"ID":"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb","Type":"ContainerStarted","Data":"e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"}
Jul 13 21:48:42.162492 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:42.162333    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" event={"ID":"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb","Type":"ContainerStarted","Data":"9f1994f4a013b5865d5a347588808bf846136d1c2be8fe2921cbb4ddcf429486"}
Jul 13 21:48:42.183027 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:42.182971    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podStartSLOduration=1.1829559490000001 podStartE2EDuration="1.182955949s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:48:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:48:42.180682061 +0000 UTC m=+2229.693431803" watchObservedRunningTime="2026-07-13 21:48:42.182955949 +0000 UTC m=+2229.695705667"
Jul 13 21:48:42.560206 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:42.560136    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:42.564453 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:42.564430    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:43.167534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:43.167500    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:43.168346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:43.168325    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:45.345341 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:48:45.345311    2564 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69ec8b7_6bc2_4efe_9453_96891f5a5db9.slice/crio-b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd69ec8b7_6bc2_4efe_9453_96891f5a5db9.slice/crio-conmon-b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632.scope\": RecentStats: unable to find data in memory cache]"
Jul 13 21:48:45.565026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.564969    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:48:45.647523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647493    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-podinfo\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647523    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-envoy\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647548    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-certs\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647584    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istiod-ca-cert\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647609    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-token\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647641    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-socket\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647703    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-data\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647733    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxr6g\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-kube-api-access-qxr6g\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.647774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647756    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-credential-socket\") pod \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\" (UID: \"d69ec8b7-6bc2-4efe-9453-96891f5a5db9\") "
Jul 13 21:48:45.648177 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.647929    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:45.648177 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.648027    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:48:45.648304 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.648196    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.648304 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.648214    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.648304 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.648269    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:45.648444 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.648374    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:45.648537 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.648512    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-data" (OuterVolumeSpecName: "istio-data") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:45.649859 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.649827    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:48:45.649975 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.649942    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-token" (OuterVolumeSpecName: "istio-token") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:48:45.650156 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.650140    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:48:45.650252 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.650232    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-kube-api-access-qxr6g" (OuterVolumeSpecName: "kube-api-access-qxr6g") pod "d69ec8b7-6bc2-4efe-9453-96891f5a5db9" (UID: "d69ec8b7-6bc2-4efe-9453-96891f5a5db9"). InnerVolumeSpecName "kube-api-access-qxr6g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:48:45.749648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749613    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.749648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749641    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.749648 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749652    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.749933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749660    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.749933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749669    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.749933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749681    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxr6g\" (UniqueName: \"kubernetes.io/projected/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-kube-api-access-qxr6g\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:45.749933 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:45.749695    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d69ec8b7-6bc2-4efe-9453-96891f5a5db9-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:48:46.185063 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.185030    2564 generic.go:356] "Generic (PLEG): container finished" podID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerID="b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632" exitCode=0
Jul 13 21:48:46.185223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.185089    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"
Jul 13 21:48:46.185223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.185092    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" event={"ID":"d69ec8b7-6bc2-4efe-9453-96891f5a5db9","Type":"ContainerDied","Data":"b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632"}
Jul 13 21:48:46.185223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.185143    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6" event={"ID":"d69ec8b7-6bc2-4efe-9453-96891f5a5db9","Type":"ContainerDied","Data":"c96606395ddeddbaa3bcbab6de966e522fed8357e5d47459be8110ebd9992098"}
Jul 13 21:48:46.185223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.185164    2564 scope.go:117] "RemoveContainer" containerID="b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632"
Jul 13 21:48:46.195007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.194992    2564 scope.go:117] "RemoveContainer" containerID="a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"
Jul 13 21:48:46.205398 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.205345    2564 scope.go:117] "RemoveContainer" containerID="b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632"
Jul 13 21:48:46.205661 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:48:46.205634    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632\": container with ID starting with b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632 not found: ID does not exist" containerID="b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632"
Jul 13 21:48:46.205764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.205665    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632"} err="failed to get container status \"b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632\": rpc error: code = NotFound desc = could not find container \"b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632\": container with ID starting with b192267c87eba57d0547301b0859cb3fddbe82aeecf08ee0c7713b6f95712632 not found: ID does not exist"
Jul 13 21:48:46.205764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.205685    2564 scope.go:117] "RemoveContainer" containerID="a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"
Jul 13 21:48:46.205938 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:48:46.205911    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578\": container with ID starting with a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578 not found: ID does not exist" containerID="a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"
Jul 13 21:48:46.206024 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.205946    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578"} err="failed to get container status \"a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578\": rpc error: code = NotFound desc = could not find container \"a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578\": container with ID starting with a6fa69bc58d4ad56272eef8facf30c19ca988bce2e36dfa031e6c68ffd442578 not found: ID does not exist"
Jul 13 21:48:46.208423 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.208395    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"]
Jul 13 21:48:46.210810 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:46.210790    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-7b2acc-openshift-default-bccbfd85c-kn7b6"]
Jul 13 21:48:47.076865 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:47.076832    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" path="/var/lib/kubelet/pods/d69ec8b7-6bc2-4efe-9453-96891f5a5db9/volumes"
Jul 13 21:48:49.212626 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.212592    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"]
Jul 13 21:48:49.213191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.213168    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy"
Jul 13 21:48:49.213191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.213191    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy"
Jul 13 21:48:49.213374 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.213353    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy"
Jul 13 21:48:49.213374 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.213375    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy"
Jul 13 21:48:49.217721 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.217697    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.221228 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.221206    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-6c68d6-openshift-default-dockercfg-ncpr6\""
Jul 13 21:48:49.229941 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.229915    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"]
Jul 13 21:48:49.281778 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.281739    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istiod-ca-cert\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.281778 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.281782    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-envoy\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.281819    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-podinfo\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.281903    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqkg7\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-kube-api-access-nqkg7\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.281935    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-credential-socket\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.281968    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-token\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282009 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.282003    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-socket\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282200 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.282073    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-data\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.282200 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.282098    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-certs\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.382709 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382667    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-envoy\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.382914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382714    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-podinfo\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.382914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382775    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqkg7\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-kube-api-access-nqkg7\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.382914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382799    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-credential-socket\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.382914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382860    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-token\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.382914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382895    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-socket\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383252 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382958    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-data\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383252 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.382985    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-certs\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383252 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.383040    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istiod-ca-cert\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383405 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.383354    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-credential-socket\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383531 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.383503    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-data\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383665 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.383550    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-socket\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383665 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.383590    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-certs\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.383791 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.383749    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istiod-ca-cert\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.385141 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.385119    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-envoy\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.385289 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.385272    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-podinfo\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.399415 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.399385    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-token\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.399551 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.399446    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqkg7\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-kube-api-access-nqkg7\") pod \"e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") " pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.529357 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.529277    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:49.671339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.671312    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"]
Jul 13 21:48:49.674767 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:48:49.674742    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20421f7b_3ab7_4382_a0bb_7e08f8e0b4c9.slice/crio-ed8edbd8dc5a864fdea3c16b4b9363a245b967768c8fca3635f573a316c1af6c WatchSource:0}: Error finding container ed8edbd8dc5a864fdea3c16b4b9363a245b967768c8fca3635f573a316c1af6c: Status 404 returned error can't find the container with id ed8edbd8dc5a864fdea3c16b4b9363a245b967768c8fca3635f573a316c1af6c
Jul 13 21:48:49.676967 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.676932    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:49.677085 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.677009    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:49.677085 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.677073    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:49.860080 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:49.858637    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:48:50.212710 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:50.212616    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" event={"ID":"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9","Type":"ContainerStarted","Data":"4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"}
Jul 13 21:48:50.212710 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:50.212672    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" event={"ID":"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9","Type":"ContainerStarted","Data":"ed8edbd8dc5a864fdea3c16b4b9363a245b967768c8fca3635f573a316c1af6c"}
Jul 13 21:48:50.234886 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:50.234834    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podStartSLOduration=1.234814746 podStartE2EDuration="1.234814746s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:48:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:48:50.23334338 +0000 UTC m=+2237.746093122" watchObservedRunningTime="2026-07-13 21:48:50.234814746 +0000 UTC m=+2237.747564486"
Jul 13 21:48:50.530340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:50.530244    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:50.532027 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:50.531997    2564 patch_prober.go:28] interesting pod/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 13 21:48:50.532216 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:50.532076    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 13 21:48:51.534152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:51.534120    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:52.230882 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:52.230846    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:52.231745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:52.231727    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:48:58.169491 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.169453    2564 patch_prober.go:28] interesting pod/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 13 21:48:58.169897 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.169508    2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 13 21:48:58.268862 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.268824    2564 generic.go:356] "Generic (PLEG): container finished" podID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerID="e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4" exitCode=0
Jul 13 21:48:58.269030 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.268883    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" event={"ID":"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb","Type":"ContainerDied","Data":"e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"}
Jul 13 21:48:58.269326 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.269313    2564 scope.go:117] "RemoveContainer" containerID="e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"
Jul 13 21:48:58.270284 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.270260    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:58.270340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.270322    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:58.270374 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:58.270360    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:48:59.276210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:59.276175    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" event={"ID":"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb","Type":"ContainerStarted","Data":"da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92"}
Jul 13 21:48:59.559675 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:59.559578    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:48:59.559849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:59.559688    2564 patch_prober.go:28] interesting pod/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 13 21:48:59.559849 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:48:59.559728    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 13 21:49:00.560214 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:00.560175    2564 patch_prober.go:28] interesting pod/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 13 21:49:00.560610 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:00.560242    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 13 21:49:01.560029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:01.559973    2564 patch_prober.go:28] interesting pod/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 13 21:49:01.560029 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:01.560031    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 13 21:49:02.559502 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:02.559461    2564 patch_prober.go:28] interesting pod/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 13 21:49:02.559714 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:02.559523    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 13 21:49:03.563193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:03.563164    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:49:03.563658 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:03.563549    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:49:03.564257 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:03.564236    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:49:05.314191 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.314155    2564 generic.go:356] "Generic (PLEG): container finished" podID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerID="4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416" exitCode=0
Jul 13 21:49:05.314697 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.314234    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" event={"ID":"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9","Type":"ContainerDied","Data":"4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"}
Jul 13 21:49:05.314946 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.314927    2564 scope.go:117] "RemoveContainer" containerID="4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"
Jul 13 21:49:05.316004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.315977    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:49:05.316106 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.316050    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:49:05.316174 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.316106    2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"}
Jul 13 21:49:05.662456 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:05.662415    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:49:06.325072 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:06.325002    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" event={"ID":"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9","Type":"ContainerStarted","Data":"f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe"}
Jul 13 21:49:06.529603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:06.529560    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:49:06.530716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:06.530690    2564 patch_prober.go:28] interesting pod/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 13 21:49:06.530836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:06.530746    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 13 21:49:07.231679 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:07.231645    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:49:07.530523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:07.530425    2564 patch_prober.go:28] interesting pod/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 13 21:49:07.530523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:07.530498    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 13 21:49:08.530404 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:08.530368    2564 patch_prober.go:28] interesting pod/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 13 21:49:08.530592 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:08.530429    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 13 21:49:09.530128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:09.530093    2564 patch_prober.go:28] interesting pod/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused" start-of-body=
Jul 13 21:49:09.530324 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:09.530158    2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.65:15021/healthz/ready\": dial tcp 10.134.0.65:15021: connect: connection refused"
Jul 13 21:49:10.533426 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:10.533392    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:49:10.534395 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:10.534375    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:49:14.954327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:14.954294    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:49:36.754441 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.754408    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"]
Jul 13 21:49:36.755044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.755009    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy"
Jul 13 21:49:36.755044 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.755044    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d69ec8b7-6bc2-4efe-9453-96891f5a5db9" containerName="istio-proxy"
Jul 13 21:49:36.759940 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.759927    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.762433 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.762412    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"openshift-service-ca.crt\""
Jul 13 21:49:36.762602 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.762583    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"kube-root-ca.crt\""
Jul 13 21:49:36.762683 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.762582    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"default-dockercfg-9s58z\""
Jul 13 21:49:36.763600 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.763581    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"auth-test-model-6c68d6-kserve-self-signed-certs\""
Jul 13 21:49:36.768514 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.768494    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"]
Jul 13 21:49:36.866358 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.866328    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-model-cache\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.866519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.866368    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-dshm\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.866519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.866407    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3c79196-995d-4b84-96f4-f013a320192d-tls-certs\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.866519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.866448    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-tmp-dir\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.866519 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.866471    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcstk\" (UniqueName: \"kubernetes.io/projected/c3c79196-995d-4b84-96f4-f013a320192d-kube-api-access-dcstk\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.866666 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.866528    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-home\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.967635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.967568    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-model-cache\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.967818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.967650    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-dshm\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.967818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.967702    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3c79196-995d-4b84-96f4-f013a320192d-tls-certs\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.967818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.967769    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-tmp-dir\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.967818 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.967805    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcstk\" (UniqueName: \"kubernetes.io/projected/c3c79196-995d-4b84-96f4-f013a320192d-kube-api-access-dcstk\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.968181 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.967841    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-home\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.968181 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.968111    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-model-cache\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.968380 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.968181    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-home\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.969765 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.968392    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-tmp-dir\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.970698 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.970655    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-dshm\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.971371 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.971348    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3c79196-995d-4b84-96f4-f013a320192d-tls-certs\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:36.976877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:36.976852    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcstk\" (UniqueName: \"kubernetes.io/projected/c3c79196-995d-4b84-96f4-f013a320192d-kube-api-access-dcstk\") pod \"auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") " pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:37.070731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:37.070704    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:37.213578 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:37.213554    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"]
Jul 13 21:49:37.215638 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:49:37.215607    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3c79196_995d_4b84_96f4_f013a320192d.slice/crio-c86a0f6e7d16f02018b5ab65790c714c5fde84119a67b0046b87f745d42bd90e WatchSource:0}: Error finding container c86a0f6e7d16f02018b5ab65790c714c5fde84119a67b0046b87f745d42bd90e: Status 404 returned error can't find the container with id c86a0f6e7d16f02018b5ab65790c714c5fde84119a67b0046b87f745d42bd90e
Jul 13 21:49:37.504925 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:37.504841    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s" event={"ID":"c3c79196-995d-4b84-96f4-f013a320192d","Type":"ContainerStarted","Data":"4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a"}
Jul 13 21:49:37.504925 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:37.504888    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s" event={"ID":"c3c79196-995d-4b84-96f4-f013a320192d","Type":"ContainerStarted","Data":"c86a0f6e7d16f02018b5ab65790c714c5fde84119a67b0046b87f745d42bd90e"}
Jul 13 21:49:37.520911 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:37.520866    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s" podStartSLOduration=1.520850705 podStartE2EDuration="1.520850705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:49:37.520032183 +0000 UTC m=+2285.032781928" watchObservedRunningTime="2026-07-13 21:49:37.520850705 +0000 UTC m=+2285.033600444"
Jul 13 21:49:40.546340 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:40.546308    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:49:47.077339 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:47.077312    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:47.077731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:47.077356    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:47.084637 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:47.084607    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:47.571691 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:47.571662    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:49:58.355189 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:49:58.355156    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:50:17.052047 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:50:17.052008    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:50:31.155710 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:50:31.155633    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:50:45.047397 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:50:45.047363    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:51:00.151334 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:00.151297    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:51:17.345461 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:17.345420    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:51:33.186101 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:33.186073    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:51:33.199724 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:33.199700    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:51:35.848624 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:35.848593    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:51:41.768095 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:41.768045    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"]
Jul 13 21:51:52.957335 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:52.957301    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:51:56.788838 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:56.788722    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s" podUID="c3c79196-995d-4b84-96f4-f013a320192d" containerName="main" containerID="cri-o://4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a" gracePeriod=45
Jul 13 21:51:57.036968 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.036944    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:51:57.059297 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059220    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3c79196-995d-4b84-96f4-f013a320192d-tls-certs\") pod \"c3c79196-995d-4b84-96f4-f013a320192d\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") "
Jul 13 21:51:57.059297 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059257    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-home\") pod \"c3c79196-995d-4b84-96f4-f013a320192d\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") "
Jul 13 21:51:57.059297 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059279    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dcstk\" (UniqueName: \"kubernetes.io/projected/c3c79196-995d-4b84-96f4-f013a320192d-kube-api-access-dcstk\") pod \"c3c79196-995d-4b84-96f4-f013a320192d\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") "
Jul 13 21:51:57.059572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059326    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-dshm\") pod \"c3c79196-995d-4b84-96f4-f013a320192d\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") "
Jul 13 21:51:57.059572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059340    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-tmp-dir\") pod \"c3c79196-995d-4b84-96f4-f013a320192d\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") "
Jul 13 21:51:57.059572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059365    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-model-cache\") pod \"c3c79196-995d-4b84-96f4-f013a320192d\" (UID: \"c3c79196-995d-4b84-96f4-f013a320192d\") "
Jul 13 21:51:57.059572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059535    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-home" (OuterVolumeSpecName: "home") pod "c3c79196-995d-4b84-96f4-f013a320192d" (UID: "c3c79196-995d-4b84-96f4-f013a320192d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:57.059784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059695    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-home\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:51:57.059784 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059766    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c3c79196-995d-4b84-96f4-f013a320192d" (UID: "c3c79196-995d-4b84-96f4-f013a320192d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:57.059892 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.059780    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-model-cache" (OuterVolumeSpecName: "model-cache") pod "c3c79196-995d-4b84-96f4-f013a320192d" (UID: "c3c79196-995d-4b84-96f4-f013a320192d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:57.061733 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.061703    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3c79196-995d-4b84-96f4-f013a320192d-kube-api-access-dcstk" (OuterVolumeSpecName: "kube-api-access-dcstk") pod "c3c79196-995d-4b84-96f4-f013a320192d" (UID: "c3c79196-995d-4b84-96f4-f013a320192d"). InnerVolumeSpecName "kube-api-access-dcstk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:51:57.061846 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.061824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-dshm" (OuterVolumeSpecName: "dshm") pod "c3c79196-995d-4b84-96f4-f013a320192d" (UID: "c3c79196-995d-4b84-96f4-f013a320192d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:51:57.061892 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.061824    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3c79196-995d-4b84-96f4-f013a320192d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c3c79196-995d-4b84-96f4-f013a320192d" (UID: "c3c79196-995d-4b84-96f4-f013a320192d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:51:57.161012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.160981    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c3c79196-995d-4b84-96f4-f013a320192d-tls-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:51:57.161012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.161009    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dcstk\" (UniqueName: \"kubernetes.io/projected/c3c79196-995d-4b84-96f4-f013a320192d-kube-api-access-dcstk\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:51:57.161012 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.161018    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-dshm\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:51:57.161260 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.161027    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-tmp-dir\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:51:57.161260 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.161038    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c3c79196-995d-4b84-96f4-f013a320192d-model-cache\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:51:57.289018 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.288979    2564 generic.go:356] "Generic (PLEG): container finished" podID="c3c79196-995d-4b84-96f4-f013a320192d" containerID="4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a" exitCode=0
Jul 13 21:51:57.289226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.289029    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"
Jul 13 21:51:57.289226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.289022    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s" event={"ID":"c3c79196-995d-4b84-96f4-f013a320192d","Type":"ContainerDied","Data":"4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a"}
Jul 13 21:51:57.289226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.289166    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s" event={"ID":"c3c79196-995d-4b84-96f4-f013a320192d","Type":"ContainerDied","Data":"c86a0f6e7d16f02018b5ab65790c714c5fde84119a67b0046b87f745d42bd90e"}
Jul 13 21:51:57.289226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.289189    2564 scope.go:117] "RemoveContainer" containerID="4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a"
Jul 13 21:51:57.298830 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.298815    2564 scope.go:117] "RemoveContainer" containerID="4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a"
Jul 13 21:51:57.299141 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:51:57.299109    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a\": container with ID starting with 4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a not found: ID does not exist" containerID="4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a"
Jul 13 21:51:57.299209 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.299185    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a"} err="failed to get container status \"4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a\": rpc error: code = NotFound desc = could not find container \"4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a\": container with ID starting with 4f276d68a819bebdbd237644c9a1b123bbacd4baec5fcccb623d50006e78dd9a not found: ID does not exist"
Jul 13 21:51:57.309499 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.309439    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"]
Jul 13 21:51:57.311802 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:57.311781    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/auth-test-model-6c68d6-kserve-5f78bf9787-wcw4s"]
Jul 13 21:51:59.076717 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:51:59.076686    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3c79196-995d-4b84-96f4-f013a320192d" path="/var/lib/kubelet/pods/c3c79196-995d-4b84-96f4-f013a320192d/volumes"
Jul 13 21:52:00.761659 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.761629    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"]
Jul 13 21:52:00.762261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.762243    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3c79196-995d-4b84-96f4-f013a320192d" containerName="main"
Jul 13 21:52:00.762261 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.762263    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3c79196-995d-4b84-96f4-f013a320192d" containerName="main"
Jul 13 21:52:00.762368 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.762359    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3c79196-995d-4b84-96f4-f013a320192d" containerName="main"
Jul 13 21:52:00.767102 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.767081    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.770774 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.770749    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"default-dockercfg-9s58z\""
Jul 13 21:52:00.770905 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.770762    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"sub-test-model-6c68d6-kserve-self-signed-certs\""
Jul 13 21:52:00.771125 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.771105    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"kube-root-ca.crt\""
Jul 13 21:52:00.771248 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.771230    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"openshift-service-ca.crt\""
Jul 13 21:52:00.774402 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.774380    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"]
Jul 13 21:52:00.894823 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.894793    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-tmp-dir\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.894998 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.894830    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8th6\" (UniqueName: \"kubernetes.io/projected/0cbac2f6-332a-4aba-9da4-08153016532e-kube-api-access-f8th6\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.894998 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.894944    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbac2f6-332a-4aba-9da4-08153016532e-tls-certs\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.894998 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.894983    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-home\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.895232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.895111    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-dshm\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.895232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.895159    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-model-cache\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.996630 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.996592    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbac2f6-332a-4aba-9da4-08153016532e-tls-certs\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.996821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.996641    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-home\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.996821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.996694    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-dshm\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.996821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.996731    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-model-cache\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.996821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.996796    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-tmp-dir\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.996821 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.996818    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8th6\" (UniqueName: \"kubernetes.io/projected/0cbac2f6-332a-4aba-9da4-08153016532e-kube-api-access-f8th6\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.997185 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.997158    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-home\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.997575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.997483    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-model-cache\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.997575 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.997508    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-tmp-dir\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.999420 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.999390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-dshm\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:00.999555 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:00.999535    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbac2f6-332a-4aba-9da4-08153016532e-tls-certs\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:01.009734 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:01.009711    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8th6\" (UniqueName: \"kubernetes.io/projected/0cbac2f6-332a-4aba-9da4-08153016532e-kube-api-access-f8th6\") pod \"sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") " pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:01.077485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:01.077458    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:01.219104 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:01.219078    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"]
Jul 13 21:52:01.220499 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:52:01.220470    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cbac2f6_332a_4aba_9da4_08153016532e.slice/crio-472e2181af4e8ab2472dba00745964cb3b2042b7069509ca588a4ec1b5326338 WatchSource:0}: Error finding container 472e2181af4e8ab2472dba00745964cb3b2042b7069509ca588a4ec1b5326338: Status 404 returned error can't find the container with id 472e2181af4e8ab2472dba00745964cb3b2042b7069509ca588a4ec1b5326338
Jul 13 21:52:01.316097 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:01.316032    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4" event={"ID":"0cbac2f6-332a-4aba-9da4-08153016532e","Type":"ContainerStarted","Data":"95a06a5656308a52d932387176b5c4a62221fafa4a38c6d1ca152171e654a7f1"}
Jul 13 21:52:01.316250 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:01.316104    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4" event={"ID":"0cbac2f6-332a-4aba-9da4-08153016532e","Type":"ContainerStarted","Data":"472e2181af4e8ab2472dba00745964cb3b2042b7069509ca588a4ec1b5326338"}
Jul 13 21:52:01.337246 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:01.337158    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4" podStartSLOduration=1.33714303 podStartE2EDuration="1.33714303s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:52:01.335488012 +0000 UTC m=+2428.848237753" watchObservedRunningTime="2026-07-13 21:52:01.33714303 +0000 UTC m=+2428.849892771"
Jul 13 21:52:06.755819 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:06.755776    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:52:11.077897 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:11.077869    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:11.078397 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:11.078001    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:11.090385 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:11.090361    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:11.384376 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:11.384291    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:52:21.948026 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:21.947990    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:52:42.760226 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:42.760186    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:52:48.475338 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:48.475304    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"]
Jul 13 21:52:59.557797 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:52:59.557758    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:53:03.498389 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.498342    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4" podUID="0cbac2f6-332a-4aba-9da4-08153016532e" containerName="main" containerID="cri-o://95a06a5656308a52d932387176b5c4a62221fafa4a38c6d1ca152171e654a7f1" gracePeriod=45
Jul 13 21:53:03.663307 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.663275    2564 generic.go:356] "Generic (PLEG): container finished" podID="0cbac2f6-332a-4aba-9da4-08153016532e" containerID="95a06a5656308a52d932387176b5c4a62221fafa4a38c6d1ca152171e654a7f1" exitCode=0
Jul 13 21:53:03.663470 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.663331    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4" event={"ID":"0cbac2f6-332a-4aba-9da4-08153016532e","Type":"ContainerDied","Data":"95a06a5656308a52d932387176b5c4a62221fafa4a38c6d1ca152171e654a7f1"}
Jul 13 21:53:03.759025 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.758973    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:53:03.825543 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825516    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8th6\" (UniqueName: \"kubernetes.io/projected/0cbac2f6-332a-4aba-9da4-08153016532e-kube-api-access-f8th6\") pod \"0cbac2f6-332a-4aba-9da4-08153016532e\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") "
Jul 13 21:53:03.825671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825558    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-home\") pod \"0cbac2f6-332a-4aba-9da4-08153016532e\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") "
Jul 13 21:53:03.825671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825648    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-tmp-dir\") pod \"0cbac2f6-332a-4aba-9da4-08153016532e\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") "
Jul 13 21:53:03.825743 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825694    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbac2f6-332a-4aba-9da4-08153016532e-tls-certs\") pod \"0cbac2f6-332a-4aba-9da4-08153016532e\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") "
Jul 13 21:53:03.825743 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825734    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-model-cache\") pod \"0cbac2f6-332a-4aba-9da4-08153016532e\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") "
Jul 13 21:53:03.825811 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825790    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-dshm\") pod \"0cbac2f6-332a-4aba-9da4-08153016532e\" (UID: \"0cbac2f6-332a-4aba-9da4-08153016532e\") "
Jul 13 21:53:03.825915 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825892    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-home" (OuterVolumeSpecName: "home") pod "0cbac2f6-332a-4aba-9da4-08153016532e" (UID: "0cbac2f6-332a-4aba-9da4-08153016532e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:03.825982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.825904    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0cbac2f6-332a-4aba-9da4-08153016532e" (UID: "0cbac2f6-332a-4aba-9da4-08153016532e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:03.826073 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.826021    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-model-cache" (OuterVolumeSpecName: "model-cache") pod "0cbac2f6-332a-4aba-9da4-08153016532e" (UID: "0cbac2f6-332a-4aba-9da4-08153016532e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:03.826231 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.826211    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-home\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.826231 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.826230    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-tmp-dir\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.826380 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.826243    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-model-cache\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.827688 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.827657    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0cbac2f6-332a-4aba-9da4-08153016532e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0cbac2f6-332a-4aba-9da4-08153016532e" (UID: "0cbac2f6-332a-4aba-9da4-08153016532e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:53:03.827767 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.827702    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-dshm" (OuterVolumeSpecName: "dshm") pod "0cbac2f6-332a-4aba-9da4-08153016532e" (UID: "0cbac2f6-332a-4aba-9da4-08153016532e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:53:03.827767 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.827707    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cbac2f6-332a-4aba-9da4-08153016532e-kube-api-access-f8th6" (OuterVolumeSpecName: "kube-api-access-f8th6") pod "0cbac2f6-332a-4aba-9da4-08153016532e" (UID: "0cbac2f6-332a-4aba-9da4-08153016532e"). InnerVolumeSpecName "kube-api-access-f8th6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:53:03.927438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.927404    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cbac2f6-332a-4aba-9da4-08153016532e-tls-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.927438 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.927433    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0cbac2f6-332a-4aba-9da4-08153016532e-dshm\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:53:03.927612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:03.927446    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8th6\" (UniqueName: \"kubernetes.io/projected/0cbac2f6-332a-4aba-9da4-08153016532e-kube-api-access-f8th6\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:53:04.670301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:04.670260    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4" event={"ID":"0cbac2f6-332a-4aba-9da4-08153016532e","Type":"ContainerDied","Data":"472e2181af4e8ab2472dba00745964cb3b2042b7069509ca588a4ec1b5326338"}
Jul 13 21:53:04.670731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:04.670316    2564 scope.go:117] "RemoveContainer" containerID="95a06a5656308a52d932387176b5c4a62221fafa4a38c6d1ca152171e654a7f1"
Jul 13 21:53:04.670731 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:04.670270    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"
Jul 13 21:53:04.693318 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:04.693294    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"]
Jul 13 21:53:04.696528 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:04.696508    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/sub-test-model-6c68d6-kserve-85cc49d6c7-8mvc4"]
Jul 13 21:53:05.077354 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:05.077327    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cbac2f6-332a-4aba-9da4-08153016532e" path="/var/lib/kubelet/pods/0cbac2f6-332a-4aba-9da4-08153016532e/volumes"
Jul 13 21:53:27.857154 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.857077    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"]
Jul 13 21:53:27.857564 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.857550    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cbac2f6-332a-4aba-9da4-08153016532e" containerName="main"
Jul 13 21:53:27.857614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.857566    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cbac2f6-332a-4aba-9da4-08153016532e" containerName="main"
Jul 13 21:53:27.857658 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.857649    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cbac2f6-332a-4aba-9da4-08153016532e" containerName="main"
Jul 13 21:53:27.860968 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.860951    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:27.864409 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.864330    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"openshift-service-ca.crt\""
Jul 13 21:53:27.864409 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.864373    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"rate-test-model-6c68d6-kserve-self-signed-certs\""
Jul 13 21:53:27.864579 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.864386    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"kube-root-ca.crt\""
Jul 13 21:53:27.864579 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.864421    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-6c68d6\"/\"default-dockercfg-9s58z\""
Jul 13 21:53:27.869153 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.869132    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"]
Jul 13 21:53:27.969075 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.969019    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-tmp-dir\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:27.969232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.969097    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-home\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:27.969232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.969126    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-dshm\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:27.969232 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.969199    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-model-cache\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:27.969341 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.969264    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rhhjs\" (UniqueName: \"kubernetes.io/projected/5ba8fb2f-82df-4093-9e21-35e4b314c829-kube-api-access-rhhjs\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:27.969341 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:27.969281    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba8fb2f-82df-4093-9e21-35e4b314c829-tls-certs\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.070638 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.070603    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-home\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.070886 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.070867    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-dshm\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.071178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.071157    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-model-cache\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.071374 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.071354    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rhhjs\" (UniqueName: \"kubernetes.io/projected/5ba8fb2f-82df-4093-9e21-35e4b314c829-kube-api-access-rhhjs\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.071510 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.071494    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba8fb2f-82df-4093-9e21-35e4b314c829-tls-certs\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.071879 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.071780    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-tmp-dir\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.072070 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.071241    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-home\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.072836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.072576    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-model-cache\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.072836 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.072790    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-tmp-dir\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.075142 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.075124    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-dshm\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.075591 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.075569    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba8fb2f-82df-4093-9e21-35e4b314c829-tls-certs\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.079867 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.079842    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rhhjs\" (UniqueName: \"kubernetes.io/projected/5ba8fb2f-82df-4093-9e21-35e4b314c829-kube-api-access-rhhjs\") pod \"rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") " pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.171435 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.171375    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:28.308276 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.308246    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"]
Jul 13 21:53:28.309396 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:53:28.309364    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ba8fb2f_82df_4093_9e21_35e4b314c829.slice/crio-4918ccc82e34ad91ca1c7fa18e28730ae970402098570a6f86040289d1b36977 WatchSource:0}: Error finding container 4918ccc82e34ad91ca1c7fa18e28730ae970402098570a6f86040289d1b36977: Status 404 returned error can't find the container with id 4918ccc82e34ad91ca1c7fa18e28730ae970402098570a6f86040289d1b36977
Jul 13 21:53:28.804018 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.803983    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r" event={"ID":"5ba8fb2f-82df-4093-9e21-35e4b314c829","Type":"ContainerStarted","Data":"002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef"}
Jul 13 21:53:28.804018 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.804023    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r" event={"ID":"5ba8fb2f-82df-4093-9e21-35e4b314c829","Type":"ContainerStarted","Data":"4918ccc82e34ad91ca1c7fa18e28730ae970402098570a6f86040289d1b36977"}
Jul 13 21:53:28.823624 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:28.823574    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r" podStartSLOduration=1.823559045 podStartE2EDuration="1.823559045s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:53:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:53:28.820870132 +0000 UTC m=+2516.333619873" watchObservedRunningTime="2026-07-13 21:53:28.823559045 +0000 UTC m=+2516.336308786"
Jul 13 21:53:32.752622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:32.752586    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:53:38.172313 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:38.172272    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:38.172745 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:38.172344    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:38.184903 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:38.184879    2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:38.871489 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:38.871458    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:53:46.170484 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:53:46.170454    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:54:00.860682 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:00.860650    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:54:08.445154 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:08.445118    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"]
Jul 13 21:54:19.156870 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:19.156835    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:54:23.465368 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.465305    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r" podUID="5ba8fb2f-82df-4093-9e21-35e4b314c829" containerName="main" containerID="cri-o://002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef" gracePeriod=45
Jul 13 21:54:23.730455 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.730392    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:54:23.815005 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.814971    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rhhjs\" (UniqueName: \"kubernetes.io/projected/5ba8fb2f-82df-4093-9e21-35e4b314c829-kube-api-access-rhhjs\") pod \"5ba8fb2f-82df-4093-9e21-35e4b314c829\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") "
Jul 13 21:54:23.815219 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815040    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba8fb2f-82df-4093-9e21-35e4b314c829-tls-certs\") pod \"5ba8fb2f-82df-4093-9e21-35e4b314c829\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") "
Jul 13 21:54:23.815219 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815092    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-tmp-dir\") pod \"5ba8fb2f-82df-4093-9e21-35e4b314c829\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") "
Jul 13 21:54:23.815219 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815118    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-dshm\") pod \"5ba8fb2f-82df-4093-9e21-35e4b314c829\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") "
Jul 13 21:54:23.815219 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815146    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-home\") pod \"5ba8fb2f-82df-4093-9e21-35e4b314c829\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") "
Jul 13 21:54:23.815219 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815162    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-model-cache\") pod \"5ba8fb2f-82df-4093-9e21-35e4b314c829\" (UID: \"5ba8fb2f-82df-4093-9e21-35e4b314c829\") "
Jul 13 21:54:23.815493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815391    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "5ba8fb2f-82df-4093-9e21-35e4b314c829" (UID: "5ba8fb2f-82df-4093-9e21-35e4b314c829"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:23.815493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815450    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-home" (OuterVolumeSpecName: "home") pod "5ba8fb2f-82df-4093-9e21-35e4b314c829" (UID: "5ba8fb2f-82df-4093-9e21-35e4b314c829"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:23.815493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815483    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-model-cache" (OuterVolumeSpecName: "model-cache") pod "5ba8fb2f-82df-4093-9e21-35e4b314c829" (UID: "5ba8fb2f-82df-4093-9e21-35e4b314c829"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:23.815613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815577    2564 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-tmp-dir\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:23.815613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815590    2564 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-home\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:23.815613 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.815600    2564 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-model-cache\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:23.817182 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.817158    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ba8fb2f-82df-4093-9e21-35e4b314c829-kube-api-access-rhhjs" (OuterVolumeSpecName: "kube-api-access-rhhjs") pod "5ba8fb2f-82df-4093-9e21-35e4b314c829" (UID: "5ba8fb2f-82df-4093-9e21-35e4b314c829"). InnerVolumeSpecName "kube-api-access-rhhjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:23.817574 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.817557    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5ba8fb2f-82df-4093-9e21-35e4b314c829-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "5ba8fb2f-82df-4093-9e21-35e4b314c829" (UID: "5ba8fb2f-82df-4093-9e21-35e4b314c829"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 13 21:54:23.817641 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.817568    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-dshm" (OuterVolumeSpecName: "dshm") pod "5ba8fb2f-82df-4093-9e21-35e4b314c829" (UID: "5ba8fb2f-82df-4093-9e21-35e4b314c829"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:23.916375 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.916342    2564 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5ba8fb2f-82df-4093-9e21-35e4b314c829-tls-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:23.916375 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.916369    2564 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5ba8fb2f-82df-4093-9e21-35e4b314c829-dshm\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:23.916375 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:23.916377    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rhhjs\" (UniqueName: \"kubernetes.io/projected/5ba8fb2f-82df-4093-9e21-35e4b314c829-kube-api-access-rhhjs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:24.114841 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.114795    2564 generic.go:356] "Generic (PLEG): container finished" podID="5ba8fb2f-82df-4093-9e21-35e4b314c829" containerID="002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef" exitCode=0
Jul 13 21:54:24.115019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.114855    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"
Jul 13 21:54:24.115019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.114874    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r" event={"ID":"5ba8fb2f-82df-4093-9e21-35e4b314c829","Type":"ContainerDied","Data":"002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef"}
Jul 13 21:54:24.115019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.114916    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r" event={"ID":"5ba8fb2f-82df-4093-9e21-35e4b314c829","Type":"ContainerDied","Data":"4918ccc82e34ad91ca1c7fa18e28730ae970402098570a6f86040289d1b36977"}
Jul 13 21:54:24.115019 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.114934    2564 scope.go:117] "RemoveContainer" containerID="002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef"
Jul 13 21:54:24.124313 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.124280    2564 scope.go:117] "RemoveContainer" containerID="002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef"
Jul 13 21:54:24.124641 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:54:24.124611    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef\": container with ID starting with 002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef not found: ID does not exist" containerID="002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef"
Jul 13 21:54:24.124699 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.124655    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef"} err="failed to get container status \"002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef\": rpc error: code = NotFound desc = could not find container \"002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef\": container with ID starting with 002a61ed00894319cadaa6d57400b6200804ffb7f536c83ced142ac35a65f7ef not found: ID does not exist"
Jul 13 21:54:24.136509 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.136484    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"]
Jul 13 21:54:24.141322 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:24.141303    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-6c68d6/rate-test-model-6c68d6-kserve-855f6bd58c-6bl9r"]
Jul 13 21:54:25.076961 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:25.076930    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ba8fb2f-82df-4093-9e21-35e4b314c829" path="/var/lib/kubelet/pods/5ba8fb2f-82df-4093-9e21-35e4b314c829/volumes"
Jul 13 21:54:41.298304 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:41.298262    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:54:53.256780 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:53.256747    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"]
Jul 13 21:54:53.257212 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:53.256941    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy" containerID="cri-o://da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92" gracePeriod=30
Jul 13 21:54:58.523736 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.523677    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:54:58.558193 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558167    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istiod-ca-cert\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558200    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-socket\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558216    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-certs\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558245    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-credential-socket\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558337 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558279    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pccq\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-kube-api-access-5pccq\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558345    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-envoy\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558389    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-podinfo\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558444    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-token\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558483    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-data\") pod \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\" (UID: \"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb\") "
Jul 13 21:54:58.558572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558496    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:58.558841 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558606    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:54:58.558841 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558621    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:58.558957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558895    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:58.558957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558897    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.558957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558934    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.558957 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.558950    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.559327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.559302    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-data" (OuterVolumeSpecName: "istio-data") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:58.560995 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.560956    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-kube-api-access-5pccq" (OuterVolumeSpecName: "kube-api-access-5pccq") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "kube-api-access-5pccq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:58.561140 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.561099    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-token" (OuterVolumeSpecName: "istio-token") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:54:58.561480 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.561460    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:54:58.561586 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.561567    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" (UID: "9a47b26b-4d2e-49cf-814b-424ab3bc1dfb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:54:58.660327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.660297    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.660327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.660321    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.660327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.660331    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.660550 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.660339    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.660550 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.660347    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:58.660550 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:58.660355    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pccq\" (UniqueName: \"kubernetes.io/projected/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb-kube-api-access-5pccq\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:54:59.315597 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.315565    2564 generic.go:356] "Generic (PLEG): container finished" podID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerID="da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92" exitCode=0
Jul 13 21:54:59.315779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.315612    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"
Jul 13 21:54:59.315779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.315652    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" event={"ID":"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb","Type":"ContainerDied","Data":"da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92"}
Jul 13 21:54:59.315779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.315705    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv" event={"ID":"9a47b26b-4d2e-49cf-814b-424ab3bc1dfb","Type":"ContainerDied","Data":"9f1994f4a013b5865d5a347588808bf846136d1c2be8fe2921cbb4ddcf429486"}
Jul 13 21:54:59.315779 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.315736    2564 scope.go:117] "RemoveContainer" containerID="da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92"
Jul 13 21:54:59.324748 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.324733    2564 scope.go:117] "RemoveContainer" containerID="e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"
Jul 13 21:54:59.333260 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.333227    2564 scope.go:117] "RemoveContainer" containerID="da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92"
Jul 13 21:54:59.333466 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:54:59.333445    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92\": container with ID starting with da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92 not found: ID does not exist" containerID="da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92"
Jul 13 21:54:59.333525 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.333472    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92"} err="failed to get container status \"da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92\": rpc error: code = NotFound desc = could not find container \"da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92\": container with ID starting with da5bbb51ee76b74e9a897bd4db772b27dab52a83bebe4b9801be6da515fa5f92 not found: ID does not exist"
Jul 13 21:54:59.333525 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.333489    2564 scope.go:117] "RemoveContainer" containerID="e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"
Jul 13 21:54:59.333741 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:54:59.333719    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4\": container with ID starting with e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4 not found: ID does not exist" containerID="e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"
Jul 13 21:54:59.333790 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.333752    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4"} err="failed to get container status \"e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4\": rpc error: code = NotFound desc = could not find container \"e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4\": container with ID starting with e54f0282c7f540622ffb90c8f5aaad4113aba8807ccafc51bb2b49a1b7b2fab4 not found: ID does not exist"
Jul 13 21:54:59.334222 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.334205    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"]
Jul 13 21:54:59.338223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:54:59.338175    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-8139bf-openshift-default-654bfb7fb7-4gslv"]
Jul 13 21:55:01.077427 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:01.077395    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" path="/var/lib/kubelet/pods/9a47b26b-4d2e-49cf-814b-424ab3bc1dfb/volumes"
Jul 13 21:55:07.464631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:07.464594    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-t7ns6"]
Jul 13 21:55:19.435680 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:19.435645    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"]
Jul 13 21:55:19.436140 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:19.435898    2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy" containerID="cri-o://f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe" gracePeriod=30
Jul 13 21:55:24.697897 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.697834    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:55:24.724368 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724333    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-token\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724389    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqkg7\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-kube-api-access-nqkg7\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724420    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-data\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724450    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-socket\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724523 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724489    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-certs\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724553    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-credential-socket\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724593    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istiod-ca-cert\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724680    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-envoy\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.724760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.724727    2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-podinfo\") pod \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\" (UID: \"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9\") "
Jul 13 21:55:24.725414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725080    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:55:24.725414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725088    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:55:24.725414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725179    2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.725414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725202    2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-workload-certs\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.725414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725273    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-data" (OuterVolumeSpecName: "istio-data") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:55:24.725414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725393    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 13 21:55:24.725829 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.725583    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:55:24.726671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.726648    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-token" (OuterVolumeSpecName: "istio-token") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:55:24.726825 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.726801    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-kube-api-access-nqkg7" (OuterVolumeSpecName: "kube-api-access-nqkg7") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "kube-api-access-nqkg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 13 21:55:24.727142 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.727119    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 13 21:55:24.727530 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.727513    2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" (UID: "20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 13 21:55:24.826327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826292    2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-podinfo\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.826327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826320    2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-token\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.826327 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826331    2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqkg7\" (UniqueName: \"kubernetes.io/projected/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-kube-api-access-nqkg7\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.826564 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826342    2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-data\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.826564 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826351    2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-credential-socket\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.826564 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826359    2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istiod-ca-cert\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:24.826564 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:24.826367    2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9-istio-envoy\") on node \"ip-10-0-131-147.ec2.internal\" DevicePath \"\""
Jul 13 21:55:25.102611 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.102580    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7cq8m_26316338-bc6d-43e8-b66f-55e96fda2f6c/manager/0.log"
Jul 13 21:55:25.223461 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.223422    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-982vc_eba5395b-ca42-4a9d-a331-3d7bdebb4708/manager/0.log"
Jul 13 21:55:25.463764 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.463670    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qrgqf_65d3467c-8fd5-4531-a2d8-619d39c47bcc/server/0.log"
Jul 13 21:55:25.469654 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.469628    2564 generic.go:356] "Generic (PLEG): container finished" podID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerID="f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe" exitCode=0
Jul 13 21:55:25.469788 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.469673    2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"
Jul 13 21:55:25.469788 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.469693    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" event={"ID":"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9","Type":"ContainerDied","Data":"f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe"}
Jul 13 21:55:25.469788 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.469735    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq" event={"ID":"20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9","Type":"ContainerDied","Data":"ed8edbd8dc5a864fdea3c16b4b9363a245b967768c8fca3635f573a316c1af6c"}
Jul 13 21:55:25.469788 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.469758    2564 scope.go:117] "RemoveContainer" containerID="f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe"
Jul 13 21:55:25.478895 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.478878    2564 scope.go:117] "RemoveContainer" containerID="4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"
Jul 13 21:55:25.487313 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.487290    2564 scope.go:117] "RemoveContainer" containerID="f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe"
Jul 13 21:55:25.487576 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:55:25.487557    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe\": container with ID starting with f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe not found: ID does not exist" containerID="f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe"
Jul 13 21:55:25.487631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.487586    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe"} err="failed to get container status \"f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe\": rpc error: code = NotFound desc = could not find container \"f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe\": container with ID starting with f4c0c1990c77ceaae438accc5343199c8053d5e096da7f978e7177c3420267fe not found: ID does not exist"
Jul 13 21:55:25.487631 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.487603    2564 scope.go:117] "RemoveContainer" containerID="4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"
Jul 13 21:55:25.487838 ip-10-0-131-147 kubenswrapper[2564]: E0713 21:55:25.487815    2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416\": container with ID starting with 4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416 not found: ID does not exist" containerID="4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"
Jul 13 21:55:25.487875 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.487848    2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416"} err="failed to get container status \"4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416\": rpc error: code = NotFound desc = could not find container \"4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416\": container with ID starting with 4bf5a862250d85152abd9ad8259af139e6b6579d785adc1654a3a10cbd7e8416 not found: ID does not exist"
Jul 13 21:55:25.487916 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.487894    2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"]
Jul 13 21:55:25.490923 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.490901    2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-6c68d6-openshift-default-fcfdc8d67-pgbxq"]
Jul 13 21:55:25.583391 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.583362    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6wpx8_5b5bee33-069b-499a-b32f-226173b22122/manager/0.log"
Jul 13 21:55:25.959672 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.959644    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf_67458df2-a147-4ca6-97d1-c4b22a2803e4/copy-manifests/0.log"
Jul 13 21:55:25.966681 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:25.966657    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf_67458df2-a147-4ca6-97d1-c4b22a2803e4/manager/0.log"
Jul 13 21:55:26.437861 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:26.437829    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29732970-2hm4s_ee7005ea-fd02-4f40-942d-daf100cf1892/cleanup/0.log"
Jul 13 21:55:26.665392 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:26.665359    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-19275bdc-4mxvc_fa905b6e-e89f-42a8-bbcf-8051bf020a2f/revoke-keys/1.log"
Jul 13 21:55:27.076921 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:27.076886    2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" path="/var/lib/kubelet/pods/20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9/volumes"
Jul 13 21:55:27.687863 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:27.687835    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-ffa5db90-xvdsp_2c1a5e2c-514a-45e7-afbb-b4759627fe13/revoke-keys/0.log"
Jul 13 21:55:29.407123 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:29.407093    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7jmsq_f1301376-d0ef-45d8-b9a8-a97237d1fbdc/manager/0.log"
Jul 13 21:55:29.524678 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:29.524655    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qj2pl_6b160ad4-2a61-44ba-8360-04d26857359e/manager/0.log"
Jul 13 21:55:29.748656 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:29.748581    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2kptt_1efc2fed-1a71-49a2-a8c2-0839a45a6619/registry-server/0.log"
Jul 13 21:55:29.870716 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:29.870684    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-75dt6_b701ac41-93fe-4670-9e37-d891c84c964f/manager/0.log"
Jul 13 21:55:29.983671 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:29.983651    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-t7ns6_8a8ad431-364b-4ffe-a252-a5dffede995e/limitador/0.log"
Jul 13 21:55:30.110936 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:30.110905    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2xrrf_93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8/manager/0.log"
Jul 13 21:55:30.477759 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:30.477685    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4z5b55_94dbfa33-448a-4b9d-987f-636cbe6e3947/istio-proxy/0.log"
Jul 13 21:55:30.826726 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:30.826698    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-hpc52_63af4ef4-2065-4c23-a017-d5db54b5c4e9/kube-auth-proxy/0.log"
Jul 13 21:55:30.964612 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:30.964581    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/1.log"
Jul 13 21:55:31.901910 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:31.901877    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-622xt_06bc429a-3247-4880-9e43-3f2ea918fbd5/main/0.log"
Jul 13 21:55:36.049387 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.049355    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59z9w/must-gather-l8lsn"]
Jul 13 21:55:36.050126 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050105    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ba8fb2f-82df-4093-9e21-35e4b314c829" containerName="main"
Jul 13 21:55:36.050126 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050127    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ba8fb2f-82df-4093-9e21-35e4b314c829" containerName="main"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050157    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050166    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050177    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050187    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050199    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050207    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050230    2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy"
Jul 13 21:55:36.050274 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050238    2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy"
Jul 13 21:55:36.050581 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050336    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy"
Jul 13 21:55:36.050581 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050347    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ba8fb2f-82df-4093-9e21-35e4b314c829" containerName="main"
Jul 13 21:55:36.050581 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050354    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy"
Jul 13 21:55:36.050581 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050360    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a47b26b-4d2e-49cf-814b-424ab3bc1dfb" containerName="istio-proxy"
Jul 13 21:55:36.050581 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.050529    2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="20421f7b-3ab7-4382-a0bb-7e08f8e0b4c9" containerName="istio-proxy"
Jul 13 21:55:36.056321 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.056298    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.059942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.059920    2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-59z9w\"/\"default-dockercfg-lj8pz\""
Jul 13 21:55:36.060112 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.060031    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59z9w\"/\"kube-root-ca.crt\""
Jul 13 21:55:36.060182 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.060161    2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-59z9w\"/\"openshift-service-ca.crt\""
Jul 13 21:55:36.063705 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.063671    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59z9w/must-gather-l8lsn"]
Jul 13 21:55:36.134550 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.134505    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cbf4e8af-6859-462f-9222-ff075cbcf396-must-gather-output\") pod \"must-gather-l8lsn\" (UID: \"cbf4e8af-6859-462f-9222-ff075cbcf396\") " pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.134756 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.134594    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsgjq\" (UniqueName: \"kubernetes.io/projected/cbf4e8af-6859-462f-9222-ff075cbcf396-kube-api-access-tsgjq\") pod \"must-gather-l8lsn\" (UID: \"cbf4e8af-6859-462f-9222-ff075cbcf396\") " pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.236045 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.236002    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cbf4e8af-6859-462f-9222-ff075cbcf396-must-gather-output\") pod \"must-gather-l8lsn\" (UID: \"cbf4e8af-6859-462f-9222-ff075cbcf396\") " pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.236254 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.236110    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsgjq\" (UniqueName: \"kubernetes.io/projected/cbf4e8af-6859-462f-9222-ff075cbcf396-kube-api-access-tsgjq\") pod \"must-gather-l8lsn\" (UID: \"cbf4e8af-6859-462f-9222-ff075cbcf396\") " pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.236412 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.236390    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/cbf4e8af-6859-462f-9222-ff075cbcf396-must-gather-output\") pod \"must-gather-l8lsn\" (UID: \"cbf4e8af-6859-462f-9222-ff075cbcf396\") " pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.244301 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.244274    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsgjq\" (UniqueName: \"kubernetes.io/projected/cbf4e8af-6859-462f-9222-ff075cbcf396-kube-api-access-tsgjq\") pod \"must-gather-l8lsn\" (UID: \"cbf4e8af-6859-462f-9222-ff075cbcf396\") " pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.368152 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.368115    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59z9w/must-gather-l8lsn"
Jul 13 21:55:36.499178 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.499153    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59z9w/must-gather-l8lsn"]
Jul 13 21:55:36.500987 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:55:36.500961    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcbf4e8af_6859_462f_9222_ff075cbcf396.slice/crio-71d631c5ac29f5849f4803a03e82e9543b2f6d86559ee4831147cae0a5fd8bc5 WatchSource:0}: Error finding container 71d631c5ac29f5849f4803a03e82e9543b2f6d86559ee4831147cae0a5fd8bc5: Status 404 returned error can't find the container with id 71d631c5ac29f5849f4803a03e82e9543b2f6d86559ee4831147cae0a5fd8bc5
Jul 13 21:55:36.503238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.503220    2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 13 21:55:36.528680 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:36.528650    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/must-gather-l8lsn" event={"ID":"cbf4e8af-6859-462f-9222-ff075cbcf396","Type":"ContainerStarted","Data":"71d631c5ac29f5849f4803a03e82e9543b2f6d86559ee4831147cae0a5fd8bc5"}
Jul 13 21:55:38.545147 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:38.545101    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/must-gather-l8lsn" event={"ID":"cbf4e8af-6859-462f-9222-ff075cbcf396","Type":"ContainerStarted","Data":"2ecb3ff570bb3f873d4e0a99473c2088510f73657c34db5ba086d7a32cb04151"}
Jul 13 21:55:38.545147 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:38.545153    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/must-gather-l8lsn" event={"ID":"cbf4e8af-6859-462f-9222-ff075cbcf396","Type":"ContainerStarted","Data":"291d2ca71a2fc1be702994014dd1221f257a3170d70f2f03fe4b2d6a8040029b"}
Jul 13 21:55:38.566080 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:38.564811    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59z9w/must-gather-l8lsn" podStartSLOduration=1.433451888 podStartE2EDuration="2.564789086s" totalImagesPullingTime="1.131337198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 21:55:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:55:38.562162251 +0000 UTC m=+2646.074911992" watchObservedRunningTime="2026-07-13 21:55:38.564789086 +0000 UTC m=+2646.077538830"
Jul 13 21:55:39.174830 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:39.174784    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-czhjv_f911bedb-e01d-4f73-a947-89e2fd905115/global-pull-secret-syncer/0.log"
Jul 13 21:55:39.298103 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:39.298067    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mr9tt_786bcfda-4260-449e-b161-d57505ba1194/konnectivity-agent/0.log"
Jul 13 21:55:39.350542 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:39.350508    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-147.ec2.internal_0ddc1535de6c0eb54f69fb18e4b2deac/haproxy/0.log"
Jul 13 21:55:44.379210 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:44.379182    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-7jmsq_f1301376-d0ef-45d8-b9a8-a97237d1fbdc/manager/0.log"
Jul 13 21:55:44.421072 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:44.421024    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-qj2pl_6b160ad4-2a61-44ba-8360-04d26857359e/manager/0.log"
Jul 13 21:55:44.526402 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:44.526366    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2kptt_1efc2fed-1a71-49a2-a8c2-0839a45a6619/registry-server/0.log"
Jul 13 21:55:44.713126 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:44.712316    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-75dt6_b701ac41-93fe-4670-9e37-d891c84c964f/manager/0.log"
Jul 13 21:55:44.748534 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:44.748504    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-t7ns6_8a8ad431-364b-4ffe-a252-a5dffede995e/limitador/0.log"
Jul 13 21:55:44.835493 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:44.835461    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-2xrrf_93f9dd3b-e5fe-4f0d-b326-2ce93bc060e8/manager/0.log"
Jul 13 21:55:46.656245 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.653009    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/alertmanager/0.log"
Jul 13 21:55:46.683669 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.683633    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/config-reloader/0.log"
Jul 13 21:55:46.715184 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.715144    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/kube-rbac-proxy-web/0.log"
Jul 13 21:55:46.735609 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.735531    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/kube-rbac-proxy/0.log"
Jul 13 21:55:46.762796 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.762764    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/kube-rbac-proxy-metric/0.log"
Jul 13 21:55:46.786760 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.786731    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/prom-label-proxy/0.log"
Jul 13 21:55:46.809180 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.809130    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_955d53bc-3343-4682-bdbc-bee9a9637ebd/init-config-reloader/0.log"
Jul 13 21:55:46.891390 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.891355    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2n2mj_abccc841-bd4a-4d6c-927b-014259705317/kube-state-metrics/0.log"
Jul 13 21:55:46.906832 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.906799    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2n2mj_abccc841-bd4a-4d6c-927b-014259705317/kube-rbac-proxy-main/0.log"
Jul 13 21:55:46.925809 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.925781    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-2n2mj_abccc841-bd4a-4d6c-927b-014259705317/kube-rbac-proxy-self/0.log"
Jul 13 21:55:46.950120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.950041    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5c9d58844f-z7jz7_b2c97d73-447f-4146-b2b8-554f40a536ad/metrics-server/0.log"
Jul 13 21:55:46.973921 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:46.973892    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-dkvks_3f481cc0-58bb-4f3f-a762-123940420847/monitoring-plugin/0.log"
Jul 13 21:55:47.080356 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.079924    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8vmk6_78c01e4b-3698-4b79-99e1-65474d804a3d/node-exporter/0.log"
Jul 13 21:55:47.107116 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.107082    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8vmk6_78c01e4b-3698-4b79-99e1-65474d804a3d/kube-rbac-proxy/0.log"
Jul 13 21:55:47.127911 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.127879    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8vmk6_78c01e4b-3698-4b79-99e1-65474d804a3d/init-textfile/0.log"
Jul 13 21:55:47.234223 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.234183    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hdplc_48ac65a8-654e-4ec2-9d05-836e9aa41865/kube-rbac-proxy-main/0.log"
Jul 13 21:55:47.254396 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.254349    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hdplc_48ac65a8-654e-4ec2-9d05-836e9aa41865/kube-rbac-proxy-self/0.log"
Jul 13 21:55:47.281942 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.281911    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-hdplc_48ac65a8-654e-4ec2-9d05-836e9aa41865/openshift-state-metrics/0.log"
Jul 13 21:55:47.339190 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.339089    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/prometheus/0.log"
Jul 13 21:55:47.358983 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.358955    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/config-reloader/0.log"
Jul 13 21:55:47.385737 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.385707    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/thanos-sidecar/0.log"
Jul 13 21:55:47.410474 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.410443    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/kube-rbac-proxy-web/0.log"
Jul 13 21:55:47.430426 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.430393    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/kube-rbac-proxy/0.log"
Jul 13 21:55:47.452513 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.452482    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/kube-rbac-proxy-thanos/0.log"
Jul 13 21:55:47.474389 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.474358    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_3e66bf19-1726-42a7-a7db-15141c2f2db3/init-config-reloader/0.log"
Jul 13 21:55:47.521077 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.517843    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-l8rbn_44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1/prometheus-operator/0.log"
Jul 13 21:55:47.544788 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.544756    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-l8rbn_44ac09b0-b3cb-47b0-823a-b6c5bfeebcf1/kube-rbac-proxy/0.log"
Jul 13 21:55:47.572674 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.572644    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-sd96x_9996cccb-036b-4b6e-ab93-7ce6b25d658c/prometheus-operator-admission-webhook/0.log"
Jul 13 21:55:47.611238 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.611164    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cb77b4fcf-hvm97_3cf7bb72-46a0-4ccf-a7d8-60a0d3311591/telemeter-client/0.log"
Jul 13 21:55:47.633539 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.633510    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cb77b4fcf-hvm97_3cf7bb72-46a0-4ccf-a7d8-60a0d3311591/reload/0.log"
Jul 13 21:55:47.655521 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.655490    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-cb77b4fcf-hvm97_3cf7bb72-46a0-4ccf-a7d8-60a0d3311591/kube-rbac-proxy/0.log"
Jul 13 21:55:47.685315 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.685283    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f5b6944bb-bb5lh_5d723943-824c-4935-8516-409e5470786e/thanos-query/0.log"
Jul 13 21:55:47.706982 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.706951    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f5b6944bb-bb5lh_5d723943-824c-4935-8516-409e5470786e/kube-rbac-proxy-web/0.log"
Jul 13 21:55:47.729987 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.729948    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f5b6944bb-bb5lh_5d723943-824c-4935-8516-409e5470786e/kube-rbac-proxy/0.log"
Jul 13 21:55:47.760258 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.760231    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f5b6944bb-bb5lh_5d723943-824c-4935-8516-409e5470786e/prom-label-proxy/0.log"
Jul 13 21:55:47.780027 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.779999    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f5b6944bb-bb5lh_5d723943-824c-4935-8516-409e5470786e/kube-rbac-proxy-rules/0.log"
Jul 13 21:55:47.801171 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:47.801142    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7f5b6944bb-bb5lh_5d723943-824c-4935-8516-409e5470786e/kube-rbac-proxy-metrics/0.log"
Jul 13 21:55:49.245890 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.245858    2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"]
Jul 13 21:55:49.253278 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.253251    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.260072 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.260035    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"]
Jul 13 21:55:49.416602 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.416557    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-sys\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.416798 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.416677    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-lib-modules\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.416798 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.416749    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6p4k\" (UniqueName: \"kubernetes.io/projected/3fea3241-6a19-47d5-9b42-2a7880dcc97d-kube-api-access-t6p4k\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.416947 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.416808    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-proc\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.416947 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.416885    2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-podres\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.518209 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518120    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-sys\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.518468 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518309    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-sys\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.518635 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518614    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-lib-modules\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.518728 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518701    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6p4k\" (UniqueName: \"kubernetes.io/projected/3fea3241-6a19-47d5-9b42-2a7880dcc97d-kube-api-access-t6p4k\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.518796 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518765    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-proc\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.518850 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518804    2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-podres\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.519004 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.518987    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-podres\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.519188 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.519139    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-proc\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.519313 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.519239    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3fea3241-6a19-47d5-9b42-2a7880dcc97d-lib-modules\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.528673 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.528645    2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6p4k\" (UniqueName: \"kubernetes.io/projected/3fea3241-6a19-47d5-9b42-2a7880dcc97d-kube-api-access-t6p4k\") pod \"perf-node-gather-daemonset-d5mz5\" (UID: \"3fea3241-6a19-47d5-9b42-2a7880dcc97d\") " pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.568508 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.568473    2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:49.780346 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:49.780315    2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"]
Jul 13 21:55:49.782035 ip-10-0-131-147 kubenswrapper[2564]: W0713 21:55:49.781995    2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3fea3241_6a19_47d5_9b42_2a7880dcc97d.slice/crio-7c8737c54b7a459fc27ac33325bc27620841c69c444cf7954bf6041ff75683d9 WatchSource:0}: Error finding container 7c8737c54b7a459fc27ac33325bc27620841c69c444cf7954bf6041ff75683d9: Status 404 returned error can't find the container with id 7c8737c54b7a459fc27ac33325bc27620841c69c444cf7954bf6041ff75683d9
Jul 13 21:55:50.122603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:50.122575    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-69969d6f4c-sjv96_5a477817-4523-4ca1-a8bc-a53d3cb05ab6/console/0.log"
Jul 13 21:55:50.657639 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:50.657600    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5" event={"ID":"3fea3241-6a19-47d5-9b42-2a7880dcc97d","Type":"ContainerStarted","Data":"23028fce32662ad2c565a0fd164e50e5659b7d850e1d5f5adea3d9e84ced39a8"}
Jul 13 21:55:50.658359 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:50.657700    2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5" event={"ID":"3fea3241-6a19-47d5-9b42-2a7880dcc97d","Type":"ContainerStarted","Data":"7c8737c54b7a459fc27ac33325bc27620841c69c444cf7954bf6041ff75683d9"}
Jul 13 21:55:50.658603 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:50.658582    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:50.676041 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:50.675985    2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5" podStartSLOduration=1.675969118 podStartE2EDuration="1.675969118s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:55:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:55:50.674648806 +0000 UTC m=+2658.187398548" watchObservedRunningTime="2026-07-13 21:55:50.675969118 +0000 UTC m=+2658.188718906"
Jul 13 21:55:51.780975 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:51.780942    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8h742_70ee3ac3-a96e-4c24-b043-653eedc55141/dns/0.log"
Jul 13 21:55:51.800144 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:51.800117    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8h742_70ee3ac3-a96e-4c24-b043-653eedc55141/kube-rbac-proxy/0.log"
Jul 13 21:55:51.930899 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:51.930870    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8z6tc_867afcc3-bfaa-457e-a67c-998900d3f853/dns-node-resolver/0.log"
Jul 13 21:55:52.523091 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:52.523034    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-qwlrm_c66b2f39-3c48-45bc-9402-f5fbe55aecef/node-ca/0.log"
Jul 13 21:55:53.423690 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:53.423653    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4z5b55_94dbfa33-448a-4b9d-987f-636cbe6e3947/istio-proxy/0.log"
Jul 13 21:55:53.586475 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:53.586445    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-hpc52_63af4ef4-2065-4c23-a017-d5db54b5c4e9/kube-auth-proxy/0.log"
Jul 13 21:55:53.615733 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:53.615703    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/0.log"
Jul 13 21:55:53.680120 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:53.680002    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/1.log"
Jul 13 21:55:54.457688 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:54.457657    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-pddgf_dce379ae-5c3e-42b8-a6d9-8b2ff403682a/serve-healthcheck-canary/0.log"
Jul 13 21:55:55.002128 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:55.002100    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7tw5q_2e425633-a728-4da0-81aa-cf53a4344a2b/kube-rbac-proxy/0.log"
Jul 13 21:55:55.027782 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:55.027753    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7tw5q_2e425633-a728-4da0-81aa-cf53a4344a2b/exporter/0.log"
Jul 13 21:55:55.047605 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:55.047576    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7tw5q_2e425633-a728-4da0-81aa-cf53a4344a2b/extractor/0.log"
Jul 13 21:55:57.118179 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.118150    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7cq8m_26316338-bc6d-43e8-b66f-55e96fda2f6c/manager/0.log"
Jul 13 21:55:57.370005 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.369913    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-982vc_eba5395b-ca42-4a9d-a331-3d7bdebb4708/manager/0.log"
Jul 13 21:55:57.496008 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.495982    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-qrgqf_65d3467c-8fd5-4531-a2d8-619d39c47bcc/server/0.log"
Jul 13 21:55:57.603466 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.603437    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-6wpx8_5b5bee33-069b-499a-b32f-226173b22122/manager/0.log"
Jul 13 21:55:57.683614 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.683531    2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-59z9w/perf-node-gather-daemonset-d5mz5"
Jul 13 21:55:57.771414 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.771386    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf_67458df2-a147-4ca6-97d1-c4b22a2803e4/manager/0.log"
Jul 13 21:55:57.789850 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:57.789828    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6bd76fcbc5-z9hvf_67458df2-a147-4ca6-97d1-c4b22a2803e4/copy-manifests/0.log"
Jul 13 21:55:59.781940 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:59.781906    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4z5b55_94dbfa33-448a-4b9d-987f-636cbe6e3947/istio-proxy/0.log"
Jul 13 21:55:59.919665 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:59.919587    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-hpc52_63af4ef4-2065-4c23-a017-d5db54b5c4e9/kube-auth-proxy/0.log"
Jul 13 21:55:59.957695 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:55:59.957663    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/0.log"
Jul 13 21:56:00.022430 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:00.022386    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/1.log"
Jul 13 21:56:01.312403 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:01.312368    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4z5b55_94dbfa33-448a-4b9d-987f-636cbe6e3947/istio-proxy/0.log"
Jul 13 21:56:01.453513 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:01.453462    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7cf89dc8f8-hpc52_63af4ef4-2065-4c23-a017-d5db54b5c4e9/kube-auth-proxy/0.log"
Jul 13 21:56:01.475353 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:01.475319    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/0.log"
Jul 13 21:56:01.544864 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:01.544823    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vgvkp_e4348c7b-31e4-41e9-abc8-f780cc694d57/istio-proxy/1.log"
Jul 13 21:56:06.044622 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.044595    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/kube-multus-additional-cni-plugins/0.log"
Jul 13 21:56:06.064470 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.064447    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/egress-router-binary-copy/0.log"
Jul 13 21:56:06.084954 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.084929    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/cni-plugins/0.log"
Jul 13 21:56:06.104039 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.104010    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/bond-cni-plugin/0.log"
Jul 13 21:56:06.125007 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.124984    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/routeoverride-cni/0.log"
Jul 13 21:56:06.145291 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.145266    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/whereabouts-cni-bincopy/0.log"
Jul 13 21:56:06.164187 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.164162    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xsk6j_91276703-6b2d-40d1-8a70-1cf7d9b78570/whereabouts-cni/0.log"
Jul 13 21:56:06.240739 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.240712    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-cwgjq_1bf9e6bf-545a-4ce9-b5ac-69d111087820/kube-multus/0.log"
Jul 13 21:56:06.306496 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.306431    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q7js4_f76ffc5f-2c98-42fb-996f-a62e56716196/network-metrics-daemon/0.log"
Jul 13 21:56:06.321795 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:06.321769    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-q7js4_f76ffc5f-2c98-42fb-996f-a62e56716196/kube-rbac-proxy/0.log"
Jul 13 21:56:07.155399 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.155340    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-controller/0.log"
Jul 13 21:56:07.184485 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.184461    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/1.log"
Jul 13 21:56:07.184666 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.184574    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovn-acl-logging/0.log"
Jul 13 21:56:07.210914 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.210889    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/kube-rbac-proxy-node/0.log"
Jul 13 21:56:07.250028 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.250002    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/kube-rbac-proxy-ovn-metrics/0.log"
Jul 13 21:56:07.292621 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.292589    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/northd/0.log"
Jul 13 21:56:07.331329 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.331298    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/nbdb/0.log"
Jul 13 21:56:07.367572 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.367543    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/sbdb/0.log"
Jul 13 21:56:07.471877 ip-10-0-131-147 kubenswrapper[2564]: I0713 21:56:07.471780    2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-6dxmp_d8d4ba3e-6aa4-4d06-9ebd-11c2ba351c40/ovnkube-controller/0.log"