Jul 29 19:27:44.077261 ip-10-0-140-87 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 29 19:27:44.077278 ip-10-0-140-87 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 29 19:27:44.077289 ip-10-0-140-87 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 29 19:27:44.077776 ip-10-0-140-87 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 29 19:27:55.350113 ip-10-0-140-87 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 29 19:27:55.350130 ip-10-0-140-87 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 01af7c863d50451a9ae22ad970f526cf --
Jul 29 19:29:37.994269 ip-10-0-140-87 systemd[1]: Starting Kubernetes Kubelet...
Jul 29 19:29:38.443012 ip-10-0-140-87 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 19:29:38.443012 ip-10-0-140-87 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 29 19:29:38.443012 ip-10-0-140-87 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 19:29:38.443012 ip-10-0-140-87 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 29 19:29:38.443012 ip-10-0-140-87 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 29 19:29:38.444520 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.444377    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 29 19:29:38.450023 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450007    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450025    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450031    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450035    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450040    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450043    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450046    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450049    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450052    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450055    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450059    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450062    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:29:38.450062 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450064    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450069    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450073    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450077    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450080    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450083    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450086    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450088    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450091    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450093    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450103    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450106    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450109    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450113    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450117    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450120    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450123    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450126    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450128    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:29:38.450378 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450131    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450133    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450136    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450138    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450141    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450143    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450146    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450148    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450150    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450153    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450155    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450157    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450160    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450162    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450165    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450168    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450171    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450174    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450177    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450179    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:29:38.450829 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450182    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450185    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450187    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450189    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450193    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450195    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450198    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450201    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450203    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450206    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450208    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450211    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450214    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450217    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450219    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450222    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450224    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450227    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450229    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450232    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:29:38.451335 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450234    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450237    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450240    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450242    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450244    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450247    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450249    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450252    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450256    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450258    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450261    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450263    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450266    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450269    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450271    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450273    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450276    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450278    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450281    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:29:38.451810 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450283    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450286    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450288    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450292    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450295    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450298    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450300    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450303    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.450306    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451044    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451051    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451054    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451057    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451060    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451063    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451065    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451068    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451071    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451074    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451077    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:29:38.452287 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451080    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451083    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451085    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451089    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451092    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451095    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451098    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451100    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451103    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451105    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451108    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451110    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451114    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451117    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451120    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451122    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451125    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451127    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451129    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451132    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:29:38.452760 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451135    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451138    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451140    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451143    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451145    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451147    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451150    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451153    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451156    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451158    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451161    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451163    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451167    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451170    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451172    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451174    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451178    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451181    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451183    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451187    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:29:38.453271 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451190    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451192    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451196    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451200    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451202    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451205    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451208    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451210    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451213    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451216    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451218    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451221    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451223    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451226    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451228    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451231    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451233    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451236    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451238    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:29:38.453758 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451241    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451244    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451247    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451249    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451252    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451254    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451257    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451261    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451264    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451267    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451270    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451273    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451276    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451279    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451282    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451284    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451287    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451289    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451291    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:29:38.454278 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451294    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451296    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451299    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451301    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451304    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451306    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451309    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451311    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451314    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451316    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451691    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451696    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451699    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451701    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451704    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451706    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451709    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451711    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451714    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451717    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:29:38.454749 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451719    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451722    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451725    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451728    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451731    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451733    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451736    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451738    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451741    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451743    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451746    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451748    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451751    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451753    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451755    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451758    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451760    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451763    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451766    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451768    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:29:38.455265 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451771    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451773    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451776    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451778    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451781    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451783    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451786    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451788    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451790    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451793    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451795    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451798    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451800    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451803    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451805    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451807    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451810    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451813    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451815    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451818    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:29:38.455772 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451821    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451823    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451826    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451828    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451831    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451834    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451836    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451838    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451841    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451843    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451846    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451848    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451851    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451853    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451857    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451859    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451862    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451864    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451866    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:29:38.456395 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451869    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451871    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451874    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451876    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451878    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451881    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451883    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451885    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451888    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451890    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451893    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451896    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451898    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451901    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451903    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451905    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451908    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451910    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451913    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:29:38.456850 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451915    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451920    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451923    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451927    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451930    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451933    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451937    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451940    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451943    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451945    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.451948    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452046    2580 flags.go:64] FLAG: --address="0.0.0.0"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452057    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452067    2580 flags.go:64] FLAG: --anonymous-auth="true"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452074    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452084    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452088    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452092    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452096    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452099    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 29 19:29:38.457350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452102    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452105    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452108    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452111    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452115    2580 flags.go:64] FLAG: --cgroup-root=""
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452118    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452121    2580 flags.go:64] FLAG: --client-ca-file=""
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452124    2580 flags.go:64] FLAG: --cloud-provider="external"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452126    2580 flags.go:64] FLAG: --cluster-dns="[]"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452130    2580 flags.go:64] FLAG: --cluster-domain=""
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452133    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452136    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452139    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452142    2580 flags.go:64] FLAG: --container-log-max-files="5"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452146    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452149    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452152    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452155    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452158    2580 flags.go:64] FLAG: --contention-profiling="false"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452160    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452163    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452167    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452170    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452174    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 29 19:29:38.457827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452176    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452179    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452182    2580 flags.go:64] FLAG: --enable-load-reader="false"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452185    2580 flags.go:64] FLAG: --enable-server="true"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452187    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452192    2580 flags.go:64] FLAG: --event-burst="100"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452195    2580 flags.go:64] FLAG: --event-qps="50"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452197    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452200    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452203    2580 flags.go:64] FLAG: --eviction-hard=""
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452207    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452209    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452213    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452218    2580 flags.go:64] FLAG: --eviction-soft=""
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452221    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452224    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452227    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452230    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452233    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452235    2580 flags.go:64] FLAG: --fail-swap-on="true"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452238    2580 flags.go:64] FLAG: --feature-gates=""
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452242    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452245    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452248    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452251    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452254    2580 flags.go:64] FLAG: --healthz-port="10248"
Jul 29 19:29:38.458443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452256    2580 flags.go:64] FLAG: --help="false"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452259    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452262    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452265    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452267    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452270    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452274    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452276    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452279    2580 flags.go:64] FLAG: --image-service-endpoint=""
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452282    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452284    2580 flags.go:64] FLAG: --kube-api-burst="100"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452287    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452290    2580 flags.go:64] FLAG: --kube-api-qps="50"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452293    2580 flags.go:64] FLAG: --kube-reserved=""
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452296    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452298    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452301    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452304    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452308    2580 flags.go:64] FLAG: --lock-file=""
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452310    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452313    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452316    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452321    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 29 19:29:38.459130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452324    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452326    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452332    2580 flags.go:64] FLAG: --logging-format="text"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452335    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452338    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452341    2580 flags.go:64] FLAG: --manifest-url=""
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452343    2580 flags.go:64] FLAG: --manifest-url-header=""
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452347    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452351    2580 flags.go:64] FLAG: --max-open-files="1000000"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452355    2580 flags.go:64] FLAG: --max-pods="110"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452358    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452361    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452364    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452366    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452369    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452372    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452375    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452381    2580 flags.go:64] FLAG: --node-status-max-images="50"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452384    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452387    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452390    2580 flags.go:64] FLAG: --pod-cidr=""
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452393    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452398    2580 flags.go:64] FLAG: --pod-manifest-path=""
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452401    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 29 19:29:38.459691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452404    2580 flags.go:64] FLAG: --pods-per-core="0"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452407    2580 flags.go:64] FLAG: --port="10250"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452410    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452415    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-040967de4b9439a3c"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452418    2580 flags.go:64] FLAG: --qos-reserved=""
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452421    2580 flags.go:64] FLAG: --read-only-port="10255"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452425    2580 flags.go:64] FLAG: --register-node="true"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452427    2580 flags.go:64] FLAG: --register-with-taints=""
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452431    2580 flags.go:64] FLAG: --registry-burst="10"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452434    2580 flags.go:64] FLAG: --registry-qps="5"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452436    2580 flags.go:64] FLAG: --reserved-cpus=""
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452440    2580 flags.go:64] FLAG: --reserved-memory=""
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452444    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452447    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452450    2580 flags.go:64] FLAG: --rotate-certificates="false"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452453    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452455    2580 flags.go:64] FLAG: --runonce="false"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452458    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452461    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452464    2580 flags.go:64] FLAG: --seccomp-default="false"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452467    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452469    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452472    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452475    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452478    2580 flags.go:64] FLAG: --storage-driver-password="root"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452481    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 29 19:29:38.460279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452484    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452487    2580 flags.go:64] FLAG: --storage-driver-user="root"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452489    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452492    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452495    2580 flags.go:64] FLAG: --system-cgroups=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452497    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452503    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452506    2580 flags.go:64] FLAG: --tls-cert-file=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452508    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452513    2580 flags.go:64] FLAG: --tls-min-version=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452516    2580 flags.go:64] FLAG: --tls-private-key-file=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452519    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452522    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452525    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452528    2580 flags.go:64] FLAG: --v="2"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452532    2580 flags.go:64] FLAG: --version="false"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452535    2580 flags.go:64] FLAG: --vmodule=""
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452541    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452544    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452658    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452663    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452667    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452670    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:29:38.460955 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452673    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452676    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452679    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452681    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452684    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452686    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452689    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452691    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452694    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452696    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452699    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452701    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452704    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452706    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452709    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452711    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452714    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452716    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452720    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452723    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:29:38.461524 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452725    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452728    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452730    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452733    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452736    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452738    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452743    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452745    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452748    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452750    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452752    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452755    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452758    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452760    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452763    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452765    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452768    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452771    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452773    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452776    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:29:38.462060 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452778    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452781    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452783    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452787    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452790    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452793    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452795    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452798    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452801    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452804    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452807    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452810    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452812    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452815    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452818    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452820    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452823    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452826    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452829    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452832    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:29:38.462553 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452835    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452838    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452840    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452842    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452845    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452847    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452850    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452852    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452855    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452858    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452860    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452862    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452865    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452867    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452870    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452872    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452876    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452879    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452882    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:29:38.463063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452885    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452887    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452890    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452893    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452896    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452898    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452901    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452903    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452906    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452908    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452911    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452916    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452919    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452921    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452924    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:29:38.463534 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.452926    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.452932    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.460477    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.460492    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.460512    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.460518    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460572    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460577    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460580    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460583    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460586    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460589    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460592    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460594    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:29:38.463910 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460598    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460600    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460603    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460606    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460608    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460611    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460613    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460616    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460619    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460622    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460624    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460627    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460631    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460634    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460637    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460640    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460642    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460645    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460647    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:29:38.464284 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460650    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460653    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460657    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460659    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460662    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460664    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460667    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460669    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460673    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460677    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460679    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460682    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460684    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460687    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460689    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460692    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460694    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460696    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460699    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460702    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:29:38.464731 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460704    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460707    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460709    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460712    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460714    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460717    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460719    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460722    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460724    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460727    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460729    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460732    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460734    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460737    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460740    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460743    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460746    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460749    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460751    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460754    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:29:38.465235 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460756    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460758    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460761    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460765    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460768    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460771    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460774    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460777    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460780    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460782    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460785    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460788    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460790    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460793    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460795    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460798    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460800    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460802    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460805    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:29:38.465718 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460807    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460810    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460813    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460815    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460818    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460820    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460823    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460826    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460828    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460831    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460834    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460837    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460839    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.460844    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 19:29:38.466203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460973    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460978    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460981    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460984    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.460987    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461007    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461011    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461013    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461016    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461018    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461021    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461024    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461026    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461028    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461031    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461033    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461036    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461038    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461041    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461043    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 29 19:29:38.466546 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461046    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461048    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461050    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461053    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461056    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461059    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461062    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461065    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461069    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461073    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461076    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461079    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461082    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461085    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461088    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461091    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461094    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461096    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461099    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 29 19:29:38.467063 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461101    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461104    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461106    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461109    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461111    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461114    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461116    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461119    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461121    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461124    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461126    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461129    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461131    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461134    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461136    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461138    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461141    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461143    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461146    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461148    2580 feature_gate.go:328] unrecognized feature gate: Example2
Jul 29 19:29:38.467540 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461151    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461153    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461156    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461159    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461161    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461164    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461166    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461168    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461171    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461173    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461176    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461178    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461181    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461183    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461185    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461188    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461190    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461193    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461196    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461198    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 29 19:29:38.468056 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461200    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461203    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461205    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461208    2580 feature_gate.go:328] unrecognized feature gate: Example
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461210    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461212    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461215    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461217    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461221    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461224    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461227    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461230    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461232    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461235    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461237    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461240    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461243    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461245    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461247    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 29 19:29:38.468542 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:38.461250    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.461254    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.461365    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.463399    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.464849    2580 server.go:1019] "Starting client certificate rotation"
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.464949    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.464980    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 29 19:29:38.469068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.467549    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 19:29:38.473191 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.473167    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 29 19:29:38.484917 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.484896    2580 log.go:25] "Validated CRI v1 runtime API"
Jul 29 19:29:38.487160 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.487143    2580 log.go:25] "Validated CRI v1 image API"
Jul 29 19:29:38.488106 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.488085    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 29 19:29:38.491758 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.491731    2580 fs.go:135] Filesystem UUIDs: map[09f77bc9-8eff-4521-9484-9499aa8b0394:/dev/nvme0n1p3 387c7961-37f6-4b46-bacd-8afb17a53315:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 29 19:29:38.491815 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.491750    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 29 19:29:38.496282 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.496181    2580 manager.go:216] Machine: {Timestamp:2026-07-29 19:29:38.495698954 +0000 UTC m=+0.373553274 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099432 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22d3bc9a0d16ff46f898074f977c30 SystemUUID:ec22d3bc-9a0d-16ff-46f8-98074f977c30 BootID:01af7c86-3d50-451a-9ae2-2ad970f526cf Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:cd:78:78:25:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:cd:78:78:25:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:3d:79:9e:d2:18 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 29 19:29:38.496282 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.496277    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 29 19:29:38.496386 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.496347    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 29 19:29:38.499041 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.499013    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 29 19:29:38.499180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.499045    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-87.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 29 19:29:38.499804 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.499795    2580 topology_manager.go:138] "Creating topology manager with none policy"
Jul 29 19:29:38.499836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.499807    2580 container_manager_linux.go:309] "Creating device plugin manager"
Jul 29 19:29:38.499836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.499814    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 19:29:38.500382 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.500365    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 19:29:38.501014 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.500986    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 29 19:29:38.503267 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.503256    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 29 19:29:38.505534 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505523    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 19:29:38.505683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505673    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 29 19:29:38.505715 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505705    2580 kubelet.go:486] "Attempting to sync node with API server"
Jul 29 19:29:38.505742 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505716    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 29 19:29:38.505742 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505730    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 29 19:29:38.505742 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505739    2580 kubelet.go:398] "Adding apiserver pod source"
Jul 29 19:29:38.505823 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.505747    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 29 19:29:38.506107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.506097    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 19:29:38.506285 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.506114    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 29 19:29:38.510126 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.510111    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 29 19:29:38.512430 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.512417    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 29 19:29:38.512479 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.512463    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 29 19:29:38.514190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514179    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 29 19:29:38.514228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514198    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 29 19:29:38.514228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514210    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 29 19:29:38.514228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514219    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 29 19:29:38.514228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514228    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514233    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514241    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514246    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514252    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514257    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514270    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514276    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514298    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 29 19:29:38.514327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.514305    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 29 19:29:38.517805 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.517763    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-87.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 29 19:29:38.517869 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.517763    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 29 19:29:38.517869 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.517808    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 29 19:29:38.518447 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.518434    2580 server.go:1268] "Started kubelet"
Jul 29 19:29:38.518572 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.518544    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 29 19:29:38.518624 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.518522    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 29 19:29:38.518624 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.518611    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 29 19:29:38.519474 ip-10-0-140-87 systemd[1]: Started Kubernetes Kubelet.
Jul 29 19:29:38.519687 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.519669    2580 server.go:310] "Adding debug handlers to kubelet server"
Jul 29 19:29:38.523936 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.523918    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 29 19:29:38.525101 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.525080    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 29 19:29:38.528362 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.527225    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-87.ec2.internal.18c6d9704521f4a9  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-87.ec2.internal,UID:ip-10-0-140-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-87.ec2.internal,},FirstTimestamp:2026-07-29 19:29:38.518398121 +0000 UTC m=+0.396252440,LastTimestamp:2026-07-29 19:29:38.518398121 +0000 UTC m=+0.396252440,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-87.ec2.internal,}"
Jul 29 19:29:38.532119 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532101    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 29 19:29:38.532201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532149    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 29 19:29:38.532201 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.532180    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:38.532808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532769    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 29 19:29:38.532808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532799    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 29 19:29:38.532941 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532828    2580 factory.go:55] Registering systemd factory
Jul 29 19:29:38.532941 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532843    2580 factory.go:223] Registration of the systemd container factory successfully
Jul 29 19:29:38.532941 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532781    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 29 19:29:38.533072 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532953    2580 reconstruct.go:97] "Volume reconstruction finished"
Jul 29 19:29:38.533072 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.532962    2580 reconciler.go:29] "Reconciler: start to sync state"
Jul 29 19:29:38.533238 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.533225    2580 factory.go:153] Registering CRI-O factory
Jul 29 19:29:38.533281 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.533244    2580 factory.go:223] Registration of the crio container factory successfully
Jul 29 19:29:38.533344 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.533292    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 29 19:29:38.533344 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.533310    2580 factory.go:103] Registering Raw factory
Jul 29 19:29:38.533344 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.533321    2580 manager.go:1196] Started watching for new ooms in manager
Jul 29 19:29:38.533457 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.533438    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 29 19:29:38.533600 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.533562    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 29 19:29:38.533919 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.533908    2580 manager.go:349] Starting recovery of all containers
Jul 29 19:29:38.544161 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.543924    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zx9bc"
Jul 29 19:29:38.544381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.544362    2580 manager.go:354] Recovery completed
Jul 29 19:29:38.547069 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.547050    2580 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Jul 29 19:29:38.550032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.550020    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.551610 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.551595    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zx9bc"
Jul 29 19:29:38.552103 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552082    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.552190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552112    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.552190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552122    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.552837 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552821    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 29 19:29:38.552837 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552833    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 29 19:29:38.553027 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552848    2580 state_mem.go:36] "Initialized new in-memory state store"
Jul 29 19:29:38.553027 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.552858    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.553633 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.553617    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.553705 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.553645    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.553705 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.553656    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.555255 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.555242    2580 policy_none.go:49] "None policy: Start"
Jul 29 19:29:38.555255 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.555258    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 29 19:29:38.555353 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.555276    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 29 19:29:38.556357 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.556343    2580 policy_none.go:47] "Start"
Jul 29 19:29:38.588542 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.588525    2580 manager.go:337] "Starting Device Plugin manager"
Jul 29 19:29:38.588628 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.588556    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 29 19:29:38.588628 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.588567    2580 server.go:87] "Starting device plugin registration server"
Jul 29 19:29:38.588821 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.588808    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 29 19:29:38.588865 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.588825    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 29 19:29:38.588955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.588941    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 29 19:29:38.589051 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.589040    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 29 19:29:38.589099 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.589051    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 29 19:29:38.589535 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.589516    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 29 19:29:38.589618 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.589567    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:38.610228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.610205    2580 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2613 into the system container \"/system.slice\": <nil>"
Jul 29 19:29:38.689485 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.689458    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.690936 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.690396    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.690936 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.690431    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.690936 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.690447    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.690936 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.690475    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.701185 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.701141    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.701185 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.701165    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-87.ec2.internal\": node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:38.707834 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.707812    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 29 19:29:38.709199 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.709172    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 29 19:29:38.709283 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.709220    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 29 19:29:38.709283 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.709243    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 29 19:29:38.709361 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.709284    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 29 19:29:38.711369 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.711350    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:29:38.742213 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.742190    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:38.810369 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.810328    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"]
Jul 29 19:29:38.810517 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.810404    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.811533 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.811514    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.811639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.811540    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.811639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.811550    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.814988 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.814974    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.815165 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.815149    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.815229 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.815189    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.815933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.815918    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.816010 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.815940    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.816010 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.815948    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.817183 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.817168    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.817312 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.817194    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.817312 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.817208    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.817722 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.817709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.817773 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.817734    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 29 19:29:38.819278 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.819262    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientMemory"
Jul 29 19:29:38.819363 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.819302    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasNoDiskPressure"
Jul 29 19:29:38.819363 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.819317    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeHasSufficientPID"
Jul 29 19:29:38.833517 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.833497    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-87.ec2.internal\" not found" node="ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.834954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.834937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab41b11aff5e6bd368de8ec2efaf8430-config\") pod \"kube-apiserver-proxy-ip-10-0-140-87.ec2.internal\" (UID: \"ab41b11aff5e6bd368de8ec2efaf8430\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.835036 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.834960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ad0b8f58a33d50b7be921e41b2d41dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal\" (UID: \"7ad0b8f58a33d50b7be921e41b2d41dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.835036 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.834976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ad0b8f58a33d50b7be921e41b2d41dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal\" (UID: \"7ad0b8f58a33d50b7be921e41b2d41dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.837240 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.837226    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-87.ec2.internal\" not found" node="ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.843265 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.843252    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:38.935526 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.935496    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab41b11aff5e6bd368de8ec2efaf8430-config\") pod \"kube-apiserver-proxy-ip-10-0-140-87.ec2.internal\" (UID: \"ab41b11aff5e6bd368de8ec2efaf8430\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.935526 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.935529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ad0b8f58a33d50b7be921e41b2d41dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal\" (UID: \"7ad0b8f58a33d50b7be921e41b2d41dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.935702 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.935550    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ad0b8f58a33d50b7be921e41b2d41dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal\" (UID: \"7ad0b8f58a33d50b7be921e41b2d41dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.935702 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.935593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ab41b11aff5e6bd368de8ec2efaf8430-config\") pod \"kube-apiserver-proxy-ip-10-0-140-87.ec2.internal\" (UID: \"ab41b11aff5e6bd368de8ec2efaf8430\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.935702 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.935614    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7ad0b8f58a33d50b7be921e41b2d41dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal\" (UID: \"7ad0b8f58a33d50b7be921e41b2d41dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.935702 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:38.935636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7ad0b8f58a33d50b7be921e41b2d41dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal\" (UID: \"7ad0b8f58a33d50b7be921e41b2d41dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:38.943937 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:38.943913    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:39.044851 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.044783    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:39.138244 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.138221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:39.141136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.141119    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:39.145172 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.145155    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:39.245703 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.245661    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:39.346245 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.346172    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-87.ec2.internal\" not found"
Jul 29 19:29:39.430182 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.430154    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:29:39.433289 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.433273    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:39.447299 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.447282    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 19:29:39.448859 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.448842    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal"
Jul 29 19:29:39.455216 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.455201    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 29 19:29:39.464350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.464325    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 29 19:29:39.464438 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.464423    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 19:29:39.464492 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.464470    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 29 19:29:39.506689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.506665    2580 apiserver.go:52] "Watching apiserver"
Jul 29 19:29:39.522518 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.522497    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 29 19:29:39.524302 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.524280    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-rx6zl","openshift-network-diagnostics/network-check-target-bmrsw","openshift-ovn-kubernetes/ovnkube-node-t6vcr","kube-system/konnectivity-agent-7fqz7","kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl","openshift-cluster-node-tuning-operator/tuned-c679b","openshift-dns/node-resolver-j5vzw","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal","openshift-multus/multus-additional-cni-plugins-w47rh","openshift-network-operator/iptables-alerter-49t24","kube-system/global-pull-secret-syncer-ffjxb","openshift-image-registry/node-ca-pg9vq","openshift-multus/multus-8lz5l"]
Jul 29 19:29:39.525925 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.525904    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.527100 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.527082    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.527630 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.527612    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.527709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.527616    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.527709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.527658    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8qhs8\""
Jul 29 19:29:39.528584 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.528568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.528824 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.528800    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.528944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.528808    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-66bzf\""
Jul 29 19:29:39.528944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.528843    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.529121 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.529038    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 29 19:29:39.529875 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.529858    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.530137 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530119    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.530521 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530503    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 29 19:29:39.530591 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530561    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.530648 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530588    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 29 19:29:39.530648 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530628    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 29 19:29:39.530723 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530651    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6vj82\""
Jul 29 19:29:39.530761 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.530732    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 29 19:29:39.531084 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.531071    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.531529 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.531509    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 29 19:29:39.531596 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.531573    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 29 19:29:39.531764 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.531751    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r7nhj\""
Jul 29 19:29:39.532221 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.532209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.532342 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.532327    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 29 19:29:39.532987 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.532971    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 29 19:29:39.533154 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533142    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 29 19:29:39.533208 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533173    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.533362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533349    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.533412 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-5qmkt\""
Jul 29 19:29:39.533512 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533497    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.533876 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533861    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 29 19:29:39.534011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.533976    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.534059 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.534026    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tkshp\""
Jul 29 19:29:39.534287 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.534274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.535012 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.534986    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.535166 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.535150    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.535362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.535350    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.535615 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.535603    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-sr9g4\""
Jul 29 19:29:39.536251 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.536233    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 29 19:29:39.536331 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.536274    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 29 19:29:39.536382 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.536342    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 29 19:29:39.536432 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.536391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-2fkcr\""
Jul 29 19:29:39.536536 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.536523    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:39.536597 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.536577    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:39.537827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.537813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.539139 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539119    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 29 19:29:39.539139 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539137    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 29 19:29:39.539271 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539211    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8r684\""
Jul 29 19:29:39.539326 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovnkube-config\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.539381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539323    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6aad7a7-132c-49e6-91ef-a32087911aee-konnectivity-ca\") pod \"konnectivity-agent-7fqz7\" (UID: \"a6aad7a7-132c-49e6-91ef-a32087911aee\") " pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.539381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-systemd\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.539381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539366    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgdrt\" (UniqueName: \"kubernetes.io/projected/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-kube-api-access-vgdrt\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.539381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539380    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539386    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovn-node-metrics-cert\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.539438    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539420    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-multus-certs\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2cpj\" (UniqueName: \"kubernetes.io/projected/f3e7c89d-0907-46ad-828d-e7bbd23183de-kube-api-access-t2cpj\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539501    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-cni-netd\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.539580 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.539526    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-os-release\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540053    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-cni-bin\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-etc-kubernetes\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540106    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysctl-d\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-iptables-alerter-script\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-var-lib-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-kubelet\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-kubelet\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-system-cni-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464c7d4e-5049-415c-bf60-3bab6ae5429c-tmp-dir\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540307    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-kubernetes\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-cni-bin\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-socket-dir-parent\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-hostroot\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540422    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/072cfb82-ea09-4696-94d8-40b44364ac05-multus-daemon-config\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540445    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-device-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.540551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-run\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540511    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5d64j\" (UniqueName: \"kubernetes.io/projected/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-kube-api-access-5d64j\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-run-netns\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540576    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-registration-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-tuned\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-systemd\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540667    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-ovn\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-socket-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/464c7d4e-5049-415c-bf60-3bab6ae5429c-hosts-file\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540749    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-conf-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrdmj\" (UniqueName: \"kubernetes.io/projected/072cfb82-ea09-4696-94d8-40b44364ac05-kube-api-access-mrdmj\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540795    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-sys-fs\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540819    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-lib-modules\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-slash\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540864    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-etc-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-env-overrides\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6aad7a7-132c-49e6-91ef-a32087911aee-agent-certs\") pod \"konnectivity-agent-7fqz7\" (UID: \"a6aad7a7-132c-49e6-91ef-a32087911aee\") " pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysconfig\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.540981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-host\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-host-slash\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541062    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-cni-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.541114    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541145    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-k8s-cni-cncf-io\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-modprobe-d\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-sys\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-log-socket\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5qd7\" (UniqueName: \"kubernetes.io/projected/027102c3-1d0d-4c79-bdeb-0099468f8beb-kube-api-access-x5qd7\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541269    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-cni-multus\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541640    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgkvg\" (UniqueName: \"kubernetes.io/projected/464c7d4e-5049-415c-bf60-3bab6ae5429c-kube-api-access-bgkvg\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovnkube-script-lib\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.541953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/072cfb82-ea09-4696-94d8-40b44364ac05-cni-binary-copy\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-etc-selinux\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541877    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-node-log\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541904    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-cnibin\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-netns\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysctl-conf\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.541985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-var-lib-kubelet\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.542031    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-tmp\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.542683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.542057    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-systemd-units\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.557490 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.557447    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-28 19:24:38 +0000 UTC" deadline="2028-04-22 15:17:57.30459907 +0000 UTC"
Jul 29 19:29:39.557490 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.557486    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15187h48m17.747115021s"
Jul 29 19:29:39.558327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.558309    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 29 19:29:39.568575 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.568557    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-wtd6t"
Jul 29 19:29:39.577276 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.577255    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-wtd6t"
Jul 29 19:29:39.634267 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.634248    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 29 19:29:39.643184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrdmj\" (UniqueName: \"kubernetes.io/projected/072cfb82-ea09-4696-94d8-40b44364ac05-kube-api-access-mrdmj\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-sys-fs\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-lib-modules\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6g54\" (UniqueName: \"kubernetes.io/projected/9bc5edc0-4b21-4f5b-9469-c73b866287fc-kube-api-access-m6g54\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-slash\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643294    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-etc-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643296    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-sys-fs\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.643317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-env-overrides\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6aad7a7-132c-49e6-91ef-a32087911aee-agent-certs\") pod \"konnectivity-agent-7fqz7\" (UID: \"a6aad7a7-132c-49e6-91ef-a32087911aee\") " pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysconfig\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-lib-modules\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-etc-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643394    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-slash\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysconfig\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643445    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-host\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643468    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-host\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-host-slash\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643515    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-cni-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-k8s-cni-cncf-io\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643550    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-host-slash\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643562    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-modprobe-d\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-sys\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643591    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-k8s-cni-cncf-io\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643514    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-host\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643611    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xf8vx\" (UniqueName: \"kubernetes.io/projected/25377374-58d1-4eab-be9b-6a35696cc589-kube-api-access-xf8vx\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:39.643645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-dbus\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-cni-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-sys\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643666    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-log-socket\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-log-socket\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5qd7\" (UniqueName: \"kubernetes.io/projected/027102c3-1d0d-4c79-bdeb-0099468f8beb-kube-api-access-x5qd7\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643705    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-cni-multus\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-modprobe-d\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgkvg\" (UniqueName: \"kubernetes.io/projected/464c7d4e-5049-415c-bf60-3bab6ae5429c-kube-api-access-bgkvg\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643871    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-cni-multus\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovnkube-script-lib\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643913    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-env-overrides\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.643917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/072cfb82-ea09-4696-94d8-40b44364ac05-cni-binary-copy\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644111    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-etc-selinux\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-etc-selinux\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-serviceca\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.644497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644261    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-node-log\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644295    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-node-log\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-cnibin\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-netns\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644354    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovnkube-script-lib\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysctl-conf\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644405    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-var-lib-kubelet\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644430    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-tmp\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-systemd-units\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysctl-conf\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/072cfb82-ea09-4696-94d8-40b44364ac05-cni-binary-copy\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644534    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-cnibin\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644540    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-netns\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovnkube-config\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-var-lib-kubelet\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644706    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6aad7a7-132c-49e6-91ef-a32087911aee-konnectivity-ca\") pod \"konnectivity-agent-7fqz7\" (UID: \"a6aad7a7-132c-49e6-91ef-a32087911aee\") " pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-systemd-units\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-systemd\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.645340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgdrt\" (UniqueName: \"kubernetes.io/projected/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-kube-api-access-vgdrt\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-os-release\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-systemd\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644871    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovn-node-metrics-cert\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-multus-certs\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2cpj\" (UniqueName: \"kubernetes.io/projected/f3e7c89d-0907-46ad-828d-e7bbd23183de-kube-api-access-t2cpj\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-run-ovn-kubernetes\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.644947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-cni-netd\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645031    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-cni-netd\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-run-multus-certs\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovnkube-config\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645172    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a6aad7a7-132c-49e6-91ef-a32087911aee-konnectivity-ca\") pod \"konnectivity-agent-7fqz7\" (UID: \"a6aad7a7-132c-49e6-91ef-a32087911aee\") " pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-os-release\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-os-release\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.646408 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-cni-bin\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-etc-kubernetes\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysctl-d\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645291    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-cni-bin\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-iptables-alerter-script\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-etc-kubernetes\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-var-lib-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-kubelet\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-sysctl-d\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-kubelet\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-var-lib-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-system-cni-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-kubelet-config\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsp2v\" (UniqueName: \"kubernetes.io/projected/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-kube-api-access-jsp2v\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645564    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-system-cni-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.647247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-host-var-lib-kubelet\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-kubelet\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464c7d4e-5049-415c-bf60-3bab6ae5429c-tmp-dir\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-kubernetes\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-cni-bin\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645770    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-cni-bin\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-socket-dir-parent\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645740    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-kubernetes\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645797    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-hostroot\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-socket-dir-parent\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/072cfb82-ea09-4696-94d8-40b44364ac05-multus-daemon-config\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645851    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-hostroot\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-device-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645883    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645892    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-iptables-alerter-script\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-run\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.648029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-device-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645953    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-system-cni-dir\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.645957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/464c7d4e-5049-415c-bf60-3bab6ae5429c-tmp-dir\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-run\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-openvswitch\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5d64j\" (UniqueName: \"kubernetes.io/projected/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-kube-api-access-5d64j\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-kubelet-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-run-netns\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-run-netns\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-registration-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-tuned\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cnibin\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646207    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-registration-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-systemd\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646235    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-ovn\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.648642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646253    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-socket-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/464c7d4e-5049-415c-bf60-3bab6ae5429c-hosts-file\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646299    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/072cfb82-ea09-4696-94d8-40b44364ac05-multus-daemon-config\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646323    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-conf-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-systemd\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/464c7d4e-5049-415c-bf60-3bab6ae5429c-hosts-file\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/027102c3-1d0d-4c79-bdeb-0099468f8beb-run-ovn\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/072cfb82-ea09-4696-94d8-40b44364ac05-multus-conf-dir\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.646448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f3e7c89d-0907-46ad-828d-e7bbd23183de-socket-dir\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.647339    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-tmp\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.647409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a6aad7a7-132c-49e6-91ef-a32087911aee-agent-certs\") pod \"konnectivity-agent-7fqz7\" (UID: \"a6aad7a7-132c-49e6-91ef-a32087911aee\") " pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.647790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/027102c3-1d0d-4c79-bdeb-0099468f8beb-ovn-node-metrics-cert\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.649138 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.648242    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-etc-tuned\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.654203 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.654185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgkvg\" (UniqueName: \"kubernetes.io/projected/464c7d4e-5049-415c-bf60-3bab6ae5429c-kube-api-access-bgkvg\") pod \"node-resolver-j5vzw\" (UID: \"464c7d4e-5049-415c-bf60-3bab6ae5429c\") " pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.654856 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.654837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrdmj\" (UniqueName: \"kubernetes.io/projected/072cfb82-ea09-4696-94d8-40b44364ac05-kube-api-access-mrdmj\") pod \"multus-8lz5l\" (UID: \"072cfb82-ea09-4696-94d8-40b44364ac05\") " pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.654954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.654916    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5qd7\" (UniqueName: \"kubernetes.io/projected/027102c3-1d0d-4c79-bdeb-0099468f8beb-kube-api-access-x5qd7\") pod \"ovnkube-node-t6vcr\" (UID: \"027102c3-1d0d-4c79-bdeb-0099468f8beb\") " pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.655419 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.655403    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5d64j\" (UniqueName: \"kubernetes.io/projected/c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23-kube-api-access-5d64j\") pod \"iptables-alerter-49t24\" (UID: \"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23\") " pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.655814 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.655800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgdrt\" (UniqueName: \"kubernetes.io/projected/bbdc2593-0edc-4592-b6e2-5cad3d1204f6-kube-api-access-vgdrt\") pod \"tuned-c679b\" (UID: \"bbdc2593-0edc-4592-b6e2-5cad3d1204f6\") " pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.656356 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.656336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2cpj\" (UniqueName: \"kubernetes.io/projected/f3e7c89d-0907-46ad-828d-e7bbd23183de-kube-api-access-t2cpj\") pod \"aws-ebs-csi-driver-node-7g9hl\" (UID: \"f3e7c89d-0907-46ad-828d-e7bbd23183de\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.718592 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.718564    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab41b11aff5e6bd368de8ec2efaf8430.slice/crio-c7a66b54f7dc084d0dda01745c06f8e59a11301e7569dd7c6d6e3f3b051fbb19 WatchSource:0}: Error finding container c7a66b54f7dc084d0dda01745c06f8e59a11301e7569dd7c6d6e3f3b051fbb19: Status 404 returned error can't find the container with id c7a66b54f7dc084d0dda01745c06f8e59a11301e7569dd7c6d6e3f3b051fbb19
Jul 29 19:29:39.718959 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.718945    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ad0b8f58a33d50b7be921e41b2d41dd.slice/crio-be8cb532692fe3ee59b4b86b76d8425e407a717dbdbb5c484a4a125e4948cf3d WatchSource:0}: Error finding container be8cb532692fe3ee59b4b86b76d8425e407a717dbdbb5c484a4a125e4948cf3d: Status 404 returned error can't find the container with id be8cb532692fe3ee59b4b86b76d8425e407a717dbdbb5c484a4a125e4948cf3d
Jul 29 19:29:39.724512 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.724494    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:29:39.747376 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.747495 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-serviceca\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.747495 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.747474    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:39.747562 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.747536    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:40.247519194 +0000 UTC m=+2.125373499 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:39.747648 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-os-release\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.747698 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.747698 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747687    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-os-release\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.747776 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.747776 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:39.747862 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.747862 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-kubelet-config\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.747862 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747815    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.747862 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsp2v\" (UniqueName: \"kubernetes.io/projected/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-kube-api-access-jsp2v\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-system-cni-dir\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-kubelet-config\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747923    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cnibin\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747927    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-serviceca\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-system-cni-dir\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cnibin\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.747982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6g54\" (UniqueName: \"kubernetes.io/projected/9bc5edc0-4b21-4f5b-9469-c73b866287fc-kube-api-access-m6g54\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-host\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.748083    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xf8vx\" (UniqueName: \"kubernetes.io/projected/25377374-58d1-4eab-be9b-6a35696cc589-kube-api-access-xf8vx\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-dbus\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.748140 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.748139    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:40.248122944 +0000 UTC m=+2.125977257 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:39.748549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-host\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.748549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748225    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-dbus\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:39.748860 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cni-binary-copy\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748926 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.748957 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.748933    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/9bc5edc0-4b21-4f5b-9469-c73b866287fc-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.758384 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.758358    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:29:39.758384 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.758376    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:29:39.758384 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.758385    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:39.758555 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:39.758427    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:40.25841684 +0000 UTC m=+2.136271146 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:39.760564 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.760543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xf8vx\" (UniqueName: \"kubernetes.io/projected/25377374-58d1-4eab-be9b-6a35696cc589-kube-api-access-xf8vx\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:39.761497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.761477    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsp2v\" (UniqueName: \"kubernetes.io/projected/4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4-kube-api-access-jsp2v\") pod \"node-ca-pg9vq\" (UID: \"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4\") " pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.761563 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.761553    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6g54\" (UniqueName: \"kubernetes.io/projected/9bc5edc0-4b21-4f5b-9469-c73b866287fc-kube-api-access-m6g54\") pod \"multus-additional-cni-plugins-w47rh\" (UID: \"9bc5edc0-4b21-4f5b-9469-c73b866287fc\") " pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.843131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.843100    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:29:39.855431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.855370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-c679b"
Jul 29 19:29:39.861274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.861253    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-49t24"
Jul 29 19:29:39.864473 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.864450    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdc2593_0edc_4592_b6e2_5cad3d1204f6.slice/crio-2f90d00649d458a32b7c7e3274801738b83b9421f3d434b668546cda89ef5ef7 WatchSource:0}: Error finding container 2f90d00649d458a32b7c7e3274801738b83b9421f3d434b668546cda89ef5ef7: Status 404 returned error can't find the container with id 2f90d00649d458a32b7c7e3274801738b83b9421f3d434b668546cda89ef5ef7
Jul 29 19:29:39.868939 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.868916    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7d5d2f1_6513_4d9d_9e6c_5563a1d14c23.slice/crio-cdaa7ac7595adf6d27bf08b4c290f22fab312540cdf685acf987c0d15fdb9852 WatchSource:0}: Error finding container cdaa7ac7595adf6d27bf08b4c290f22fab312540cdf685acf987c0d15fdb9852: Status 404 returned error can't find the container with id cdaa7ac7595adf6d27bf08b4c290f22fab312540cdf685acf987c0d15fdb9852
Jul 29 19:29:39.892708 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.892679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:29:39.898722 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.898706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:29:39.900739 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.900706    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod027102c3_1d0d_4c79_bdeb_0099468f8beb.slice/crio-63dd17cbd5a871e8b2fbf3e6b5db46f74a2eb5eb68508259fd682f25881a864a WatchSource:0}: Error finding container 63dd17cbd5a871e8b2fbf3e6b5db46f74a2eb5eb68508259fd682f25881a864a: Status 404 returned error can't find the container with id 63dd17cbd5a871e8b2fbf3e6b5db46f74a2eb5eb68508259fd682f25881a864a
Jul 29 19:29:39.906071 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.906051    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6aad7a7_132c_49e6_91ef_a32087911aee.slice/crio-24269d55e252eb119ff3fbe7dd69f68728d71224867fdafcfb66383626613830 WatchSource:0}: Error finding container 24269d55e252eb119ff3fbe7dd69f68728d71224867fdafcfb66383626613830: Status 404 returned error can't find the container with id 24269d55e252eb119ff3fbe7dd69f68728d71224867fdafcfb66383626613830
Jul 29 19:29:39.914914 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.914897    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-8lz5l"
Jul 29 19:29:39.922131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.922114    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl"
Jul 29 19:29:39.922494 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.922475    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod072cfb82_ea09_4696_94d8_40b44364ac05.slice/crio-523e530c9ff299842f38daf99d2c57d7d82e6a2c14474ec3f314f0ce0a38be9a WatchSource:0}: Error finding container 523e530c9ff299842f38daf99d2c57d7d82e6a2c14474ec3f314f0ce0a38be9a: Status 404 returned error can't find the container with id 523e530c9ff299842f38daf99d2c57d7d82e6a2c14474ec3f314f0ce0a38be9a
Jul 29 19:29:39.929754 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.929733    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e7c89d_0907_46ad_828d_e7bbd23183de.slice/crio-33b5641122ab98a0f85108564c9f0832b1a6aa6655e56b5393a192443eaaf904 WatchSource:0}: Error finding container 33b5641122ab98a0f85108564c9f0832b1a6aa6655e56b5393a192443eaaf904: Status 404 returned error can't find the container with id 33b5641122ab98a0f85108564c9f0832b1a6aa6655e56b5393a192443eaaf904
Jul 29 19:29:39.936853 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.936839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j5vzw"
Jul 29 19:29:39.943767 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.943745    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod464c7d4e_5049_415c_bf60_3bab6ae5429c.slice/crio-b4edde6479be5a6a17dbed98636acc3dab2706636c33faec1f1cd0fb67745be7 WatchSource:0}: Error finding container b4edde6479be5a6a17dbed98636acc3dab2706636c33faec1f1cd0fb67745be7: Status 404 returned error can't find the container with id b4edde6479be5a6a17dbed98636acc3dab2706636c33faec1f1cd0fb67745be7
Jul 29 19:29:39.964060 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.964036    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-pg9vq"
Jul 29 19:29:39.969099 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:39.969074    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w47rh"
Jul 29 19:29:39.971371 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.971351    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ffbdef9_85a9_4272_b1ba_dbe67cb24dd4.slice/crio-95363b1bf267ebabb8042b75d87841d506c92e35b80a843087dbbbc368e7ec46 WatchSource:0}: Error finding container 95363b1bf267ebabb8042b75d87841d506c92e35b80a843087dbbbc368e7ec46: Status 404 returned error can't find the container with id 95363b1bf267ebabb8042b75d87841d506c92e35b80a843087dbbbc368e7ec46
Jul 29 19:29:39.976247 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:29:39.976193    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc5edc0_4b21_4f5b_9469_c73b866287fc.slice/crio-74193d2766adc885de8132ea1d27760e93e97b40dec6988a880a8d15029fe6df WatchSource:0}: Error finding container 74193d2766adc885de8132ea1d27760e93e97b40dec6988a880a8d15029fe6df: Status 404 returned error can't find the container with id 74193d2766adc885de8132ea1d27760e93e97b40dec6988a880a8d15029fe6df
Jul 29 19:29:40.040392 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.040360    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:29:40.250786 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.250756    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:40.250925 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.250801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:40.250925 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.250896    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:40.250925 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.250904    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:40.251106 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.250949    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:41.250930301 +0000 UTC m=+3.128784605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:40.251106 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.250973    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:41.250965517 +0000 UTC m=+3.128819824 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:40.351755 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.351720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:40.351975 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.351897    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:29:40.351975 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.351916    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:29:40.351975 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.351928    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:40.352152 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:40.352036    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:41.351985274 +0000 UTC m=+3.229839593 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:40.579335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.579248    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 19:24:39 +0000 UTC" deadline="2028-03-17 03:36:52.87850844 +0000 UTC"
Jul 29 19:29:40.579335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.579283    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14312h7m12.299229323s"
Jul 29 19:29:40.598260 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.598066    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:29:40.745766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.745577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"63dd17cbd5a871e8b2fbf3e6b5db46f74a2eb5eb68508259fd682f25881a864a"}
Jul 29 19:29:40.754041 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.753876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49t24" event={"ID":"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23","Type":"ContainerStarted","Data":"cdaa7ac7595adf6d27bf08b4c290f22fab312540cdf685acf987c0d15fdb9852"}
Jul 29 19:29:40.767610 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.767575    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c679b" event={"ID":"bbdc2593-0edc-4592-b6e2-5cad3d1204f6","Type":"ContainerStarted","Data":"2f90d00649d458a32b7c7e3274801738b83b9421f3d434b668546cda89ef5ef7"}
Jul 29 19:29:40.773048 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.772977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal" event={"ID":"7ad0b8f58a33d50b7be921e41b2d41dd","Type":"ContainerStarted","Data":"be8cb532692fe3ee59b4b86b76d8425e407a717dbdbb5c484a4a125e4948cf3d"}
Jul 29 19:29:40.778565 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.778537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal" event={"ID":"ab41b11aff5e6bd368de8ec2efaf8430","Type":"ContainerStarted","Data":"c7a66b54f7dc084d0dda01745c06f8e59a11301e7569dd7c6d6e3f3b051fbb19"}
Jul 29 19:29:40.786020 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.785966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerStarted","Data":"74193d2766adc885de8132ea1d27760e93e97b40dec6988a880a8d15029fe6df"}
Jul 29 19:29:40.793390 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.793315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pg9vq" event={"ID":"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4","Type":"ContainerStarted","Data":"95363b1bf267ebabb8042b75d87841d506c92e35b80a843087dbbbc368e7ec46"}
Jul 29 19:29:40.801767 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.801729    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j5vzw" event={"ID":"464c7d4e-5049-415c-bf60-3bab6ae5429c","Type":"ContainerStarted","Data":"b4edde6479be5a6a17dbed98636acc3dab2706636c33faec1f1cd0fb67745be7"}
Jul 29 19:29:40.816848 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.816788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl" event={"ID":"f3e7c89d-0907-46ad-828d-e7bbd23183de","Type":"ContainerStarted","Data":"33b5641122ab98a0f85108564c9f0832b1a6aa6655e56b5393a192443eaaf904"}
Jul 29 19:29:40.838015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.837909    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lz5l" event={"ID":"072cfb82-ea09-4696-94d8-40b44364ac05","Type":"ContainerStarted","Data":"523e530c9ff299842f38daf99d2c57d7d82e6a2c14474ec3f314f0ce0a38be9a"}
Jul 29 19:29:40.843895 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:40.843863    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7fqz7" event={"ID":"a6aad7a7-132c-49e6-91ef-a32087911aee","Type":"ContainerStarted","Data":"24269d55e252eb119ff3fbe7dd69f68728d71224867fdafcfb66383626613830"}
Jul 29 19:29:41.265191 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.265071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:41.265191 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.265133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:41.265396 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.265281    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:41.265396 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.265344    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:43.265325375 +0000 UTC m=+5.143179683 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:41.265756 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.265737    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:41.265848 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.265792    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:43.265776384 +0000 UTC m=+5.143630693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:41.366290 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.365415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:41.366290 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.365633    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:29:41.366290 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.365654    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:29:41.366290 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.365666    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:41.366290 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.365729    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:43.365711901 +0000 UTC m=+5.243566210 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:41.579502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.579398    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-28 19:24:39 +0000 UTC" deadline="2028-05-11 07:54:56.655191834 +0000 UTC"
Jul 29 19:29:41.579502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.579460    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15636h25m15.075734938s"
Jul 29 19:29:41.709892 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.709859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:41.710091 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.710012    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:41.710422 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.710403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:41.710600 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.710581    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:41.710677 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:41.710649    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:41.710727 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:41.710715    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:42.575519 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:42.575269    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 29 19:29:43.282749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:43.282119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:43.282749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:43.282160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:43.282749 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.282309    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:43.282749 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.282359    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:47.282345328 +0000 UTC m=+9.160199634 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:43.282749 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.282673    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:43.282749 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.282713    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:47.282698002 +0000 UTC m=+9.160552308 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:43.383418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:43.383380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:43.383631 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.383607    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:29:43.383699 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.383638    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:29:43.383699 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.383653    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:43.383812 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.383718    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:47.383698425 +0000 UTC m=+9.261552737 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:43.710012 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:43.709916    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:43.710163 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.710082    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:43.710453 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:43.710435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:43.710559 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.710539    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:43.710633 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:43.710593    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:43.710697 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:43.710679    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:45.710178 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:45.710143    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:45.710600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:45.710182    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:45.710600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:45.710188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:45.710600 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:45.710297    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:45.710600 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:45.710415    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:45.710600 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:45.710494    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:47.319339 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:47.319293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:47.319751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:47.319364    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:47.319751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.319478    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:47.319751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.319554    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:55.319532926 +0000 UTC m=+17.197387238 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:47.319751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.319553    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:47.319751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.319619    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:55.319600825 +0000 UTC m=+17.197455130 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:47.420286 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:47.420175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:47.420460 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.420354    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:29:47.420460 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.420379    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:29:47.420460 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.420401    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:47.420609 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.420458    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:29:55.420440896 +0000 UTC m=+17.298295208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:47.710083 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:47.710010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:47.710219 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.710147    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:47.710503 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:47.710487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:47.710575 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.710560    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:47.710610 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:47.710605    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:47.710666 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:47.710654    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:49.710133 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:49.710095    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:49.710133 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:49.710131    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:49.710611 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:49.710234    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:49.710611 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:49.710274    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:49.710611 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:49.710312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:49.710611 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:49.710389    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:51.710268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:51.710186    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:51.710268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:51.710213    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:51.710738 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:51.710311    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:51.710738 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:51.710324    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:51.710738 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:51.710455    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:51.710738 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:51.710542    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:53.710372 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:53.710331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:53.710372 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:53.710364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:53.716912 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:53.710331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:53.716912 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:53.710462    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:53.716912 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:53.710585    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:53.716912 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:53.710695    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:55.377245 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:55.377208    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:55.377691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:55.377305    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:55.377691 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.377356    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:55.377691 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.377413    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:11.377394331 +0000 UTC m=+33.255248637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:29:55.377691 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.377426    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:55.377691 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.377475    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:11.377459256 +0000 UTC m=+33.255313576 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:29:55.477927 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:55.477881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:55.478118 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.478044    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:29:55.478118 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.478061    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:29:55.478118 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.478070    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:55.478118 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.478117    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:11.478104052 +0000 UTC m=+33.355958358 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:29:55.710420 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:55.710336    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:55.710420 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:55.710388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:55.710698 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:55.710348    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:55.710698 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.710454    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:55.710698 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.710557    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:55.710698 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:55.710636    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:57.709436 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:57.709399    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:57.709882 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:57.709514    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:57.709882 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:57.709522    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:57.709882 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:57.709520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:57.709882 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:57.709603    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:57.709882 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:57.709691    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:59.710334 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.710299    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:29:59.710722 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.710393    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:29:59.710722 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:59.710403    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:29:59.710722 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.710419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:29:59.710722 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:59.710496    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:29:59.710722 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:29:59.710560    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:29:59.895237 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.895211    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-c679b" event={"ID":"bbdc2593-0edc-4592-b6e2-5cad3d1204f6","Type":"ContainerStarted","Data":"9bbc2917afbfef0c64cdd7fd45ed9eb6c344b79b93eb14372f2cdba1a036d79e"}
Jul 29 19:29:59.896593 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.896569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal" event={"ID":"ab41b11aff5e6bd368de8ec2efaf8430","Type":"ContainerStarted","Data":"d3fadd65ae78b6e1c39d572208ec7aa776a690d859125643d2dabec32f3af9f6"}
Jul 29 19:29:59.922385 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.922343    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-c679b" podStartSLOduration=2.112009215 podStartE2EDuration="21.922329351s" totalImagesPullingTime="19.810320136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:29:59.921965984 +0000 UTC m=+21.799820312" watchObservedRunningTime="2026-07-29 19:29:59.922329351 +0000 UTC m=+21.800183696"
Jul 29 19:29:59.935985 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:29:59.935943    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-87.ec2.internal" podStartSLOduration=20.93592952 podStartE2EDuration="20.93592952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:29:59.935289728 +0000 UTC m=+21.813144058" watchObservedRunningTime="2026-07-29 19:29:59.93592952 +0000 UTC m=+21.813783845"
Jul 29 19:30:00.900429 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.900030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-8lz5l" event={"ID":"072cfb82-ea09-4696-94d8-40b44364ac05","Type":"ContainerStarted","Data":"b5d3ba0da5af79ad53a9b931000c2f951b03ec023d42be5af1f48cb814de0c1f"}
Jul 29 19:30:00.902019 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.901971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7fqz7" event={"ID":"a6aad7a7-132c-49e6-91ef-a32087911aee","Type":"ContainerStarted","Data":"6e20598e4fbf11b0e9b2568baa2876c1f811ea089b0ac1bbb85135aed3e84d61"}
Jul 29 19:30:00.904279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904264    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:30:00.904611 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904589    2580 generic.go:356] "Generic (PLEG): container finished" podID="027102c3-1d0d-4c79-bdeb-0099468f8beb" containerID="2dd910d094e5d8e73f8fc8d63dbb959a0101fda6372684388177de2d43603d64" exitCode=1
Jul 29 19:30:00.904692 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904654    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"ad56dfb94bb80c15d3b572c5befa28bcdddc1e14c3ceaabf19f0b34c3428457b"}
Jul 29 19:30:00.904733 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904724    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"09c718b4c160021d965e4b085672a49b4b944308ff6c556c2f4f421bd13290fd"}
Jul 29 19:30:00.904790 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904776    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"5e867482a2cdbe6253da15c8ca57e5e225da48bdd20cc4d38d5a72fe2eb51277"}
Jul 29 19:30:00.904848 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904829    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerDied","Data":"2dd910d094e5d8e73f8fc8d63dbb959a0101fda6372684388177de2d43603d64"}
Jul 29 19:30:00.904934 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.904874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"a0ac1c855b27a25e37ce2bcef3feaa25d68bf781de8c9b1414dac432cb9eb3f4"}
Jul 29 19:30:00.905970 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.905953    2580 generic.go:356] "Generic (PLEG): container finished" podID="7ad0b8f58a33d50b7be921e41b2d41dd" containerID="50deb1a6e78811457da8a622a7b029091229d89945d5ba064765e9bf438d51bb" exitCode=0
Jul 29 19:30:00.906102 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.906028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal" event={"ID":"7ad0b8f58a33d50b7be921e41b2d41dd","Type":"ContainerDied","Data":"50deb1a6e78811457da8a622a7b029091229d89945d5ba064765e9bf438d51bb"}
Jul 29 19:30:00.907396 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.907377    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc5edc0-4b21-4f5b-9469-c73b866287fc" containerID="c3b6275df2e4790dccdbe4ebb2f802f6b4e00a5e431b0d4f383785598932edd5" exitCode=0
Jul 29 19:30:00.907471 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.907450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerDied","Data":"c3b6275df2e4790dccdbe4ebb2f802f6b4e00a5e431b0d4f383785598932edd5"}
Jul 29 19:30:00.908790 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.908713    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-pg9vq" event={"ID":"4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4","Type":"ContainerStarted","Data":"6ff7952f32a6a349d61dbb5579f08fba8c04ed2f7dbcf31928838e4369eeaed4"}
Jul 29 19:30:00.909960 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.909942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j5vzw" event={"ID":"464c7d4e-5049-415c-bf60-3bab6ae5429c","Type":"ContainerStarted","Data":"9a9fc8089b7668ba626fad951d4c9c0474eb3f2598a77d14812bb5681c920f10"}
Jul 29 19:30:00.913476 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.913450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl" event={"ID":"f3e7c89d-0907-46ad-828d-e7bbd23183de","Type":"ContainerStarted","Data":"4d90e71c7e9bd0a14f4b109c0e51d15ec9658711404728fcf6804f625865ed0c"}
Jul 29 19:30:00.922430 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.922381    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-8lz5l" podStartSLOduration=3.16988432 podStartE2EDuration="22.922368136s" totalImagesPullingTime="19.752483816s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:00.921591132 +0000 UTC m=+22.799445460" watchObservedRunningTime="2026-07-29 19:30:00.922368136 +0000 UTC m=+22.800222463"
Jul 29 19:30:00.983662 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.981971    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7fqz7" podStartSLOduration=3.265710493 podStartE2EDuration="22.981951251s" totalImagesPullingTime="19.716240758s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:00.965656623 +0000 UTC m=+22.843510950" watchObservedRunningTime="2026-07-29 19:30:00.981951251 +0000 UTC m=+22.859805582"
Jul 29 19:30:00.999069 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.999016    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-pg9vq" podStartSLOduration=3.293257629 podStartE2EDuration="22.998973126s" totalImagesPullingTime="19.705715497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:00.998947544 +0000 UTC m=+22.876801886" watchObservedRunningTime="2026-07-29 19:30:00.998973126 +0000 UTC m=+22.876827455"
Jul 29 19:30:00.999749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:00.999719    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-j5vzw" podStartSLOduration=3.265367819 podStartE2EDuration="22.999707654s" totalImagesPullingTime="19.734339835s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:00.983685255 +0000 UTC m=+22.861539584" watchObservedRunningTime="2026-07-29 19:30:00.999707654 +0000 UTC m=+22.877561984"
Jul 29 19:30:01.156744 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.156718    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:30:01.157436 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.157417    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:30:01.710132 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.709961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:01.710132 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.709968    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:01.710132 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.710106    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:01.710132 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:01.710106    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:01.710418 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:01.710189    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:01.710418 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:01.710273    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:01.793359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.793332    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 29 19:30:01.917873 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.917838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl" event={"ID":"f3e7c89d-0907-46ad-828d-e7bbd23183de","Type":"ContainerStarted","Data":"1a1248e180e547e0ff1612d2411b73b97bfb93f98c2a80aea0560144c6b4745e"}
Jul 29 19:30:01.920951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.920930    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:30:01.921553 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.921524    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"c32f05922673ae08e36ba7cc98481ddb60f3a298d31e22cff50bfc21258570ac"}
Jul 29 19:30:01.923073 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.923012    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-49t24" event={"ID":"c7d5d2f1-6513-4d9d-9e6c-5563a1d14c23","Type":"ContainerStarted","Data":"9e68f840633408f127f9da503d192e3dbd55c7b6d7aee856f7f3d635aba388cb"}
Jul 29 19:30:01.924151 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.924126    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:30:01.924646 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.924627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7fqz7"
Jul 29 19:30:01.938353 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:01.938307    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-49t24" podStartSLOduration=4.185167884 podStartE2EDuration="23.938287458s" totalImagesPullingTime="19.753119574s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:01.937486545 +0000 UTC m=+23.815340873" watchObservedRunningTime="2026-07-29 19:30:01.938287458 +0000 UTC m=+23.816141831"
Jul 29 19:30:02.603050 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:02.602914    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-29T19:30:01.793354429Z","UUID":"9e79c58a-430c-4b1f-8bbf-09d8351dc70b","Handler":null,"Name":"","Endpoint":""}
Jul 29 19:30:02.604676 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:02.604653    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 29 19:30:02.604823 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:02.604697    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 29 19:30:02.927427 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:02.927343    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal" event={"ID":"7ad0b8f58a33d50b7be921e41b2d41dd","Type":"ContainerStarted","Data":"0efb00720edd73b9dc87f8e999c934307c20ce307f1a3a2e9b3a3e398d38901b"}
Jul 29 19:30:02.942266 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:02.942221    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-87.ec2.internal" podStartSLOduration=23.942205722 podStartE2EDuration="23.942205722s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:02.941516973 +0000 UTC m=+24.819371306" watchObservedRunningTime="2026-07-29 19:30:02.942205722 +0000 UTC m=+24.820060052"
Jul 29 19:30:03.709698 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.709505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:03.709963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.709505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:03.709963 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:03.709786    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:03.709963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.709505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:03.709963 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:03.709882    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:03.710166 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:03.709962    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:03.932918 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.932892    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:30:03.933499 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.933355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"467f931f4f3b6f4819f998117d4525ee8318820f492a31f3de879addf42a6e96"}
Jul 29 19:30:03.935569 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.935535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl" event={"ID":"f3e7c89d-0907-46ad-828d-e7bbd23183de","Type":"ContainerStarted","Data":"6325f72b36bc028a9ffdcf9f055b448fe5f4b934389966d38296d3dc4186ce93"}
Jul 29 19:30:03.954777 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:03.954733    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-7g9hl" podStartSLOduration=3.649455155 podStartE2EDuration="25.954715273s" totalImagesPullingTime="22.305260118s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:03.954186439 +0000 UTC m=+25.832040769" watchObservedRunningTime="2026-07-29 19:30:03.954715273 +0000 UTC m=+25.832569614"
Jul 29 19:30:05.710565 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.710406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:05.711106 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.710411    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:05.711106 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:05.710647    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:05.711106 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.710419    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:05.711106 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:05.710720    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:05.711106 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:05.710826    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:05.943533 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.943497    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc5edc0-4b21-4f5b-9469-c73b866287fc" containerID="342794a66dde373c8d5b24b148aec0e145a93742e2e5c1fd2e0cb44762de9241" exitCode=0
Jul 29 19:30:05.943709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.943545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerDied","Data":"342794a66dde373c8d5b24b148aec0e145a93742e2e5c1fd2e0cb44762de9241"}
Jul 29 19:30:05.946987 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.946968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:30:05.947326 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.947298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"9922e93905478e61e54108c0f24ba85482904d8a852ca5ac1fcb94d6591eecf0"}
Jul 29 19:30:05.947559 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.947543    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:30:05.947744 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.947728    2580 scope.go:117] "RemoveContainer" containerID="2dd910d094e5d8e73f8fc8d63dbb959a0101fda6372684388177de2d43603d64"
Jul 29 19:30:05.966784 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:05.966548    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:30:06.952944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:06.952912    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:30:06.953463 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:06.953288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" event={"ID":"027102c3-1d0d-4c79-bdeb-0099468f8beb","Type":"ContainerStarted","Data":"73d6641352ad0a01d942c2831bb84226a8e7c493cc9ef91d903ba3615322b0d8"}
Jul 29 19:30:06.953463 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:06.953430    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 19:30:06.953704 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:06.953689    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:30:06.969600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:06.969574    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:30:07.018303 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.018252    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr" podStartSLOduration=8.677818046 podStartE2EDuration="29.018232923s" totalImagesPullingTime="20.340414877s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:06.98157298 +0000 UTC m=+28.859427307" watchObservedRunningTime="2026-07-29 19:30:07.018232923 +0000 UTC m=+28.896087250"
Jul 29 19:30:07.709904 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.709871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:07.710123 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.709975    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:07.710123 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.710020    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:07.710123 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:07.710083    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:07.710123 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:07.709978    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:07.710339 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:07.710179    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:07.717154 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.717125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffjxb"]
Jul 29 19:30:07.719574 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.719546    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bmrsw"]
Jul 29 19:30:07.728307 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.728284    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rx6zl"]
Jul 29 19:30:07.957684 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.957648    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc5edc0-4b21-4f5b-9469-c73b866287fc" containerID="e208515d640f3c0969427bab8dc5d8f8bc07a0bfda583ce5360cfe41bc3b8861" exitCode=0
Jul 29 19:30:07.958169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.957740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:07.958169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.957764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerDied","Data":"e208515d640f3c0969427bab8dc5d8f8bc07a0bfda583ce5360cfe41bc3b8861"}
Jul 29 19:30:07.958169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.957811    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:07.958169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.957837    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 29 19:30:07.958169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:07.957901    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:07.958169 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:07.957925    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:07.958441 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:07.958308    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:07.958441 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:07.958398    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:08.249631 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:08.249558    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:30:09.709874 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:09.709839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:09.710274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:09.709880    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:09.710274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:09.709881    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:09.710274 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:09.710014    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:09.710274 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:09.710021    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:09.710274 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:09.710071    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:09.968135 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:09.968051    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc5edc0-4b21-4f5b-9469-c73b866287fc" containerID="e3a8515e56ee9767a84f7c356fd2177edcc31ec792a26fa68a844b64269973a1" exitCode=0
Jul 29 19:30:09.968135 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:09.968114    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerDied","Data":"e3a8515e56ee9767a84f7c356fd2177edcc31ec792a26fa68a844b64269973a1"}
Jul 29 19:30:11.395638 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:11.395605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:11.396183 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:11.395664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:11.396183 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.395763    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:30:11.396183 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.395797    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 29 19:30:11.396183 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.395862    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret podName:6b8bf68c-0fd8-442c-af0b-90e8fbe11f84 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:43.395843285 +0000 UTC m=+65.273697593 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret") pod "global-pull-secret-syncer-ffjxb" (UID: "6b8bf68c-0fd8-442c-af0b-90e8fbe11f84") : object "kube-system"/"original-pull-secret" not registered
Jul 29 19:30:11.396183 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.395880    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:43.395870797 +0000 UTC m=+65.273725109 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 29 19:30:11.497049 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:11.496976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:11.497225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.497153    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 29 19:30:11.497225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.497179    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 29 19:30:11.497225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.497193    2580 projected.go:196] Error preparing data for projected volume kube-api-access-q8whn for pod openshift-network-diagnostics/network-check-target-bmrsw: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:30:11.497392 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.497272    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn podName:7264fa1c-f617-43fd-a8d7-cfe79f7fc782 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:43.497250629 +0000 UTC m=+65.375104941 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-q8whn" (UniqueName: "kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn") pod "network-check-target-bmrsw" (UID: "7264fa1c-f617-43fd-a8d7-cfe79f7fc782") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 29 19:30:11.709979 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:11.709894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:11.710151 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:11.710015    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:11.710151 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.710033    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:11.710151 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:11.710043    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:11.710151 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.710125    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:11.710341 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:11.710199    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:13.709747 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.709705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:13.710413 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.709815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:13.710413 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:13.709827    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ffjxb" podUID="6b8bf68c-0fd8-442c-af0b-90e8fbe11f84"
Jul 29 19:30:13.710413 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.709850    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:13.710413 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:13.709950    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bmrsw" podUID="7264fa1c-f617-43fd-a8d7-cfe79f7fc782"
Jul 29 19:30:13.710413 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:13.710054    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rx6zl" podUID="25377374-58d1-4eab-be9b-6a35696cc589"
Jul 29 19:30:13.941924 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.941892    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-87.ec2.internal" event="NodeReady"
Jul 29 19:30:13.942104 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.942087    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 29 19:30:13.981552 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.981519    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-54fcb899b6-zz94f"]
Jul 29 19:30:13.983271 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.983249    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:13.985037 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.985014    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 29 19:30:13.985123 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.985021    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 29 19:30:13.985417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.985397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 29 19:30:13.985499 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.985438    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-95c74\""
Jul 29 19:30:13.989344 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:13.989322    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 29 19:30:14.000607 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.000582    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54fcb899b6-zz94f"]
Jul 29 19:30:14.003847 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.003821    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pclnw"]
Jul 29 19:30:14.005483 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.005465    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-n88rq"]
Jul 29 19:30:14.005620 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.005604    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.006951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.006933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.007314 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.007297    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 29 19:30:14.007420 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.007299    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 29 19:30:14.007569 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.007555    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8z4sg\""
Jul 29 19:30:14.008568 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.008548    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ck8zl\""
Jul 29 19:30:14.008662 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.008622    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 29 19:30:14.008662 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.008648    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 29 19:30:14.008662 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.008649    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 29 19:30:14.018663 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.018642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n88rq"]
Jul 29 19:30:14.026651 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.026630    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pclnw"]
Jul 29 19:30:14.118306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-certificates\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118456 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-installation-pull-secrets\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118456 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b0bec5-445e-4fa4-bde1-86becceb24a8-cert\") pod \"ingress-canary-n88rq\" (UID: \"e3b0bec5-445e-4fa4-bde1-86becceb24a8\") " pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.118456 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-bound-sa-token\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118456 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f66716-3a4b-4276-bef5-7575195acbfc-tmp-dir\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.118614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118525    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04ff50d9-45ce-4220-80e6-9a042447bde1-ca-trust-extracted\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118676 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psph2\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-kube-api-access-psph2\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118676 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118656    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9f66716-3a4b-4276-bef5-7575195acbfc-config-volume\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.118745 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-image-registry-private-configuration\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.118745 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118722    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.118813 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmt6n\" (UniqueName: \"kubernetes.io/projected/f9f66716-3a4b-4276-bef5-7575195acbfc-kube-api-access-qmt6n\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.118813 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l8jt\" (UniqueName: \"kubernetes.io/projected/e3b0bec5-445e-4fa4-bde1-86becceb24a8-kube-api-access-2l8jt\") pod \"ingress-canary-n88rq\" (UID: \"e3b0bec5-445e-4fa4-bde1-86becceb24a8\") " pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.118894 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.118826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-trusted-ca\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.219743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psph2\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-kube-api-access-psph2\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.219743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9f66716-3a4b-4276-bef5-7575195acbfc-config-volume\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.219921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-image-registry-private-configuration\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.219921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.219921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219856    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmt6n\" (UniqueName: \"kubernetes.io/projected/f9f66716-3a4b-4276-bef5-7575195acbfc-kube-api-access-qmt6n\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.219921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219879    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l8jt\" (UniqueName: \"kubernetes.io/projected/e3b0bec5-445e-4fa4-bde1-86becceb24a8-kube-api-access-2l8jt\") pod \"ingress-canary-n88rq\" (UID: \"e3b0bec5-445e-4fa4-bde1-86becceb24a8\") " pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.219921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-trusted-ca\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-certificates\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.219980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-installation-pull-secrets\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b0bec5-445e-4fa4-bde1-86becceb24a8-cert\") pod \"ingress-canary-n88rq\" (UID: \"e3b0bec5-445e-4fa4-bde1-86becceb24a8\") " pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-bound-sa-token\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f66716-3a4b-4276-bef5-7575195acbfc-tmp-dir\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.220193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220142    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04ff50d9-45ce-4220-80e6-9a042447bde1-ca-trust-extracted\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.220530 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f9f66716-3a4b-4276-bef5-7575195acbfc-config-volume\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.220699    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-certificates\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.220725    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fcb899b6-zz94f: secret "image-registry-tls" not found
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.220767    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.220786    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls podName:04ff50d9-45ce-4220-80e6-9a042447bde1 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:14.720769368 +0000 UTC m=+36.598623674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls") pod "image-registry-54fcb899b6-zz94f" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1") : secret "image-registry-tls" not found
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.220835    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls podName:f9f66716-3a4b-4276-bef5-7575195acbfc nodeName:}" failed. No retries permitted until 2026-07-29 19:30:14.720815889 +0000 UTC m=+36.598670195 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls") pod "dns-default-pclnw" (UID: "f9f66716-3a4b-4276-bef5-7575195acbfc") : secret "dns-default-metrics-tls" not found
Jul 29 19:30:14.221013 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.220944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f9f66716-3a4b-4276-bef5-7575195acbfc-tmp-dir\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.221444 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.221170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04ff50d9-45ce-4220-80e6-9a042447bde1-ca-trust-extracted\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.221628 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.221608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-trusted-ca\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.224474 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.224446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-installation-pull-secrets\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.224556 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.224524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-image-registry-private-configuration\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.224609 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.224593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e3b0bec5-445e-4fa4-bde1-86becceb24a8-cert\") pod \"ingress-canary-n88rq\" (UID: \"e3b0bec5-445e-4fa4-bde1-86becceb24a8\") " pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.229127 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.229106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:14.235926 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.235896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmt6n\" (UniqueName: \"kubernetes.io/projected/f9f66716-3a4b-4276-bef5-7575195acbfc-kube-api-access-qmt6n\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.236061 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.235957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-bound-sa-token\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.236470 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.236452    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psph2\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-kube-api-access-psph2\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.236545 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.236510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l8jt\" (UniqueName: \"kubernetes.io/projected/e3b0bec5-445e-4fa4-bde1-86becceb24a8-kube-api-access-2l8jt\") pod \"ingress-canary-n88rq\" (UID: \"e3b0bec5-445e-4fa4-bde1-86becceb24a8\") " pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.323159 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.323125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:14.323941 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.323924    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-n88rq"
Jul 29 19:30:14.458312 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.458053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-n88rq"]
Jul 29 19:30:14.461601 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:14.461575    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3b0bec5_445e_4fa4_bde1_86becceb24a8.slice/crio-ae0bf2681d3359b2eb799b9449fc9331e7a89d6f17c9280eec81fdc6ef9f3678 WatchSource:0}: Error finding container ae0bf2681d3359b2eb799b9449fc9331e7a89d6f17c9280eec81fdc6ef9f3678: Status 404 returned error can't find the container with id ae0bf2681d3359b2eb799b9449fc9331e7a89d6f17c9280eec81fdc6ef9f3678
Jul 29 19:30:14.638064 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.638031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:14.724306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.724219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:14.724306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.724284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:14.724903 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.724388    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 19:30:14.724903 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.724393    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 19:30:14.724903 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.724414    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fcb899b6-zz94f: secret "image-registry-tls" not found
Jul 29 19:30:14.724903 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.724466    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls podName:f9f66716-3a4b-4276-bef5-7575195acbfc nodeName:}" failed. No retries permitted until 2026-07-29 19:30:15.724442753 +0000 UTC m=+37.602297066 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls") pod "dns-default-pclnw" (UID: "f9f66716-3a4b-4276-bef5-7575195acbfc") : secret "dns-default-metrics-tls" not found
Jul 29 19:30:14.724903 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:14.724485    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls podName:04ff50d9-45ce-4220-80e6-9a042447bde1 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:15.724476891 +0000 UTC m=+37.602331197 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls") pod "image-registry-54fcb899b6-zz94f" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1") : secret "image-registry-tls" not found
Jul 29 19:30:14.735815 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.735791    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:14.837173 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.837143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:14.981866 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.981766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n88rq" event={"ID":"e3b0bec5-445e-4fa4-bde1-86becceb24a8","Type":"ContainerStarted","Data":"ae0bf2681d3359b2eb799b9449fc9331e7a89d6f17c9280eec81fdc6ef9f3678"}
Jul 29 19:30:14.984146 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.984125    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc5edc0-4b21-4f5b-9469-c73b866287fc" containerID="be18148bc1b0d3dac73fe8b2f2d5662ee8b96ebe5ee24d496728e540bed184d1" exitCode=0
Jul 29 19:30:14.984250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:14.984165    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerDied","Data":"be18148bc1b0d3dac73fe8b2f2d5662ee8b96ebe5ee24d496728e540bed184d1"}
Jul 29 19:30:15.440969 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.440929    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:15.540329 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.540299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:15.642218 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.642186    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:15.710349 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.710276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:15.710349 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.710320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:15.710541 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.710468    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:15.712597 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.712574    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 29 19:30:15.712887 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.712867    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 19:30:15.713120 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.713102    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 29 19:30:15.713208 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.713105    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 19:30:15.713208 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.713187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xr6nn\""
Jul 29 19:30:15.713415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.713398    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qwcp5\""
Jul 29 19:30:15.720468 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.720447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:15.734717 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.734694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:15.735069 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.734752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:15.735069 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:15.734873    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 19:30:15.735069 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:15.734907    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 19:30:15.735069 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:15.734924    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls podName:f9f66716-3a4b-4276-bef5-7575195acbfc nodeName:}" failed. No retries permitted until 2026-07-29 19:30:17.734909482 +0000 UTC m=+39.612763788 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls") pod "dns-default-pclnw" (UID: "f9f66716-3a4b-4276-bef5-7575195acbfc") : secret "dns-default-metrics-tls" not found
Jul 29 19:30:15.735069 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:15.734925    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fcb899b6-zz94f: secret "image-registry-tls" not found
Jul 29 19:30:15.735069 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:15.734976    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls podName:04ff50d9-45ce-4220-80e6-9a042447bde1 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:17.734960241 +0000 UTC m=+39.612814553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls") pod "image-registry-54fcb899b6-zz94f" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1") : secret "image-registry-tls" not found
Jul 29 19:30:15.991815 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.991673    2580 generic.go:356] "Generic (PLEG): container finished" podID="9bc5edc0-4b21-4f5b-9469-c73b866287fc" containerID="0b997ee7de8668cb24502056092cc3ea3f16e7675583c8cd5906b0199eee307a" exitCode=0
Jul 29 19:30:15.991815 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:15.991757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerDied","Data":"0b997ee7de8668cb24502056092cc3ea3f16e7675583c8cd5906b0199eee307a"}
Jul 29 19:30:16.569358 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:16.569153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:16.938212 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:16.938166    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:16.997083 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:16.997049    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w47rh" event={"ID":"9bc5edc0-4b21-4f5b-9469-c73b866287fc","Type":"ContainerStarted","Data":"5a737b11b3a97a7582992dd6cfcfac7e24b09ae7f19bdee1e9a5087b04db9637"}
Jul 29 19:30:17.020404 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.020356    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:17.040505 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.040456    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w47rh" podStartSLOduration=8.472871847 podStartE2EDuration="39.040440859s" totalImagesPullingTime="30.567569012s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:17.039626442 +0000 UTC m=+38.917480771" watchObservedRunningTime="2026-07-29 19:30:17.040440859 +0000 UTC m=+38.918295187"
Jul 29 19:30:17.119254 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.119214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:17.231299 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.231255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:17.322553 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.322512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:17.752056 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.752017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:17.752210 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:17.752089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:17.752210 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:17.752160    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 19:30:17.752210 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:17.752192    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 19:30:17.752210 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:17.752202    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fcb899b6-zz94f: secret "image-registry-tls" not found
Jul 29 19:30:17.752356 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:17.752239    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls podName:f9f66716-3a4b-4276-bef5-7575195acbfc nodeName:}" failed. No retries permitted until 2026-07-29 19:30:21.752221614 +0000 UTC m=+43.630075920 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls") pod "dns-default-pclnw" (UID: "f9f66716-3a4b-4276-bef5-7575195acbfc") : secret "dns-default-metrics-tls" not found
Jul 29 19:30:17.752356 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:17.752254    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls podName:04ff50d9-45ce-4220-80e6-9a042447bde1 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:21.752248058 +0000 UTC m=+43.630102364 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls") pod "image-registry-54fcb899b6-zz94f" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1") : secret "image-registry-tls" not found
Jul 29 19:30:18.000495 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:18.000461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-n88rq" event={"ID":"e3b0bec5-445e-4fa4-bde1-86becceb24a8","Type":"ContainerStarted","Data":"56f4ccdaa4cfe98c0c13ee42c0522490e80f8396408605e5f11750ad69489818"}
Jul 29 19:30:18.020772 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:18.020674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:18.037151 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:18.037104    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-n88rq" podStartSLOduration=2.30541342 podStartE2EDuration="5.037087918s" totalImagesPullingTime="2.731674498s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:18.036510714 +0000 UTC m=+39.914365042" watchObservedRunningTime="2026-07-29 19:30:18.037087918 +0000 UTC m=+39.914942243"
Jul 29 19:30:18.409144 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:18.409102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:18.482476 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:18.482443    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:20.834915 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:20.834878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:21.539601 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.539568    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"]
Jul 29 19:30:21.565711 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.565680    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"]
Jul 29 19:30:21.565844 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.565807    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"
Jul 29 19:30:21.567685 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.567670    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-4wlrb\""
Jul 29 19:30:21.627012 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.626966    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"]
Jul 29 19:30:21.644169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.644134    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-jrc7p"]
Jul 29 19:30:21.644387 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.644357    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"
Jul 29 19:30:21.646332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.646305    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.646451 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.646390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-cgvcv\""
Jul 29 19:30:21.646451 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.646318    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.670343 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.670307    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-dlrtg"]
Jul 29 19:30:21.670515 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.670452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.672639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.672614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 29 19:30:21.672639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.672632    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.672808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.672623    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 29 19:30:21.672808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.672770    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-j4vvr\""
Jul 29 19:30:21.672926 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.672772    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.678468 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.678446    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 29 19:30:21.684268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.684249    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"]
Jul 29 19:30:21.684387 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.684373    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.684786 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.684762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hb97\" (UniqueName: \"kubernetes.io/projected/9203ac2b-f4c8-453a-b5d5-aff9e4b5d125-kube-api-access-5hb97\") pod \"network-check-source-7d88cb55d8-zkz64\" (UID: \"9203ac2b-f4c8-453a-b5d5-aff9e4b5d125\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"
Jul 29 19:30:21.686201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.686187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 29 19:30:21.686324 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.686308    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7gwdm\""
Jul 29 19:30:21.686373 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.686363    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.686442 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.686427    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 29 19:30:21.686530 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.686455    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.691862 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.691839    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 29 19:30:21.706433 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.706410    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-66df6dd487-2zjcx"]
Jul 29 19:30:21.706556 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.706543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:21.708523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.708500    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-56md8\""
Jul 29 19:30:21.708782 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.708763    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.708916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.708898    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.709210 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.709192    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 29 19:30:21.730511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730487    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"]
Jul 29 19:30:21.730645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-jrc7p"]
Jul 29 19:30:21.730645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730548    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-dlrtg"]
Jul 29 19:30:21.730645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730572    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"]
Jul 29 19:30:21.730645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730603    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66df6dd487-2zjcx"]
Jul 29 19:30:21.730645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730630    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"]
Jul 29 19:30:21.730786 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.730603    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.732373 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732339    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.732508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732381    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 29 19:30:21.732508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732396    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 29 19:30:21.732508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732421    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 29 19:30:21.732508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732486    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 29 19:30:21.732707 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732674    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.732756 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.732735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b9qrq\""
Jul 29 19:30:21.746699 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.746678    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.748392 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.748370    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 29 19:30:21.748392 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.748379    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.748559 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.748380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.748559 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.748507    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6d2fs\""
Jul 29 19:30:21.748834 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.748819    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 29 19:30:21.756637 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.756614    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"]
Jul 29 19:30:21.766445 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.766421    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"]
Jul 29 19:30:21.785793 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/588349d1-be21-4927-b936-0ba5f6e725b3-tmp\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.785793 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785704    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/588349d1-be21-4927-b936-0ba5f6e725b3-serving-cert\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.785793 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxp5g\" (UniqueName: \"kubernetes.io/projected/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-kube-api-access-vxp5g\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hb97\" (UniqueName: \"kubernetes.io/projected/9203ac2b-f4c8-453a-b5d5-aff9e4b5d125-kube-api-access-5hb97\") pod \"network-check-source-7d88cb55d8-zkz64\" (UID: \"9203ac2b-f4c8-453a-b5d5-aff9e4b5d125\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785868    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-serving-cert\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq9sf\" (UniqueName: \"kubernetes.io/projected/342114ba-5b4a-43af-9260-78a7e1013d93-kube-api-access-zq9sf\") pod \"volume-data-source-validator-549c6d54cf-clg2w\" (UID: \"342114ba-5b4a-43af-9260-78a7e1013d93\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.785980    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/588349d1-be21-4927-b936-0ba5f6e725b3-snapshots\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-trusted-ca\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-config\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786102    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqgrz\" (UniqueName: \"kubernetes.io/projected/588349d1-be21-4927-b936-0ba5f6e725b3-kube-api-access-qqgrz\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.786201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/588349d1-be21-4927-b936-0ba5f6e725b3-service-ca-bundle\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.786664 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786257    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/588349d1-be21-4927-b936-0ba5f6e725b3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.786664 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.786288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:21.786664 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.786411    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 19:30:21.786664 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.786425    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fcb899b6-zz94f: secret "image-registry-tls" not found
Jul 29 19:30:21.786664 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.786474    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls podName:04ff50d9-45ce-4220-80e6-9a042447bde1 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:29.78645757 +0000 UTC m=+51.664311879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls") pod "image-registry-54fcb899b6-zz94f" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1") : secret "image-registry-tls" not found
Jul 29 19:30:21.786664 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.786580    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 19:30:21.787032 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.786745    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls podName:f9f66716-3a4b-4276-bef5-7575195acbfc nodeName:}" failed. No retries permitted until 2026-07-29 19:30:29.786671607 +0000 UTC m=+51.664525919 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls") pod "dns-default-pclnw" (UID: "f9f66716-3a4b-4276-bef5-7575195acbfc") : secret "dns-default-metrics-tls" not found
Jul 29 19:30:21.787032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.787021    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"]
Jul 29 19:30:21.787130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.787100    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.788709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.788689    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 29 19:30:21.788959 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.788937    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.788959 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.788956    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.789194 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.789176    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 29 19:30:21.789261 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.789241    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-xqbxg\""
Jul 29 19:30:21.805753 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.805730    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hb97\" (UniqueName: \"kubernetes.io/projected/9203ac2b-f4c8-453a-b5d5-aff9e4b5d125-kube-api-access-5hb97\") pod \"network-check-source-7d88cb55d8-zkz64\" (UID: \"9203ac2b-f4c8-453a-b5d5-aff9e4b5d125\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"
Jul 29 19:30:21.807113 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.807095    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"]
Jul 29 19:30:21.807207 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.807132    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"]
Jul 29 19:30:21.807207 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.807194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:21.808678 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.808660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 29 19:30:21.808781 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.808669    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 29 19:30:21.808781 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.808753    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:30:21.808781 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.808761    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 29 19:30:21.808781 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.808774    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kbfc4\""
Jul 29 19:30:21.874689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.874646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"
Jul 29 19:30:21.886976 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.886947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-config\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.887054 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.886985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/efb86fd8-4782-4e7d-b732-52987e85c001-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.887104 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:21.887218 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7894863b-2223-4519-b749-851b66facc7b-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.887267 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/588349d1-be21-4927-b936-0ba5f6e725b3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.887303 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.887352 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-default-certificate\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.887398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/588349d1-be21-4927-b936-0ba5f6e725b3-snapshots\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.887398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887379    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-trusted-ca\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.887512 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj7w4\" (UniqueName: \"kubernetes.io/projected/efb86fd8-4782-4e7d-b732-52987e85c001-kube-api-access-zj7w4\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.887564 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.887606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqgrz\" (UniqueName: \"kubernetes.io/projected/588349d1-be21-4927-b936-0ba5f6e725b3-kube-api-access-qqgrz\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.887606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887584    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-stats-auth\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.887691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-config\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.887691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887658    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/588349d1-be21-4927-b936-0ba5f6e725b3-service-ca-bundle\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.887783 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7894863b-2223-4519-b749-851b66facc7b-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.887783 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887735    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/588349d1-be21-4927-b936-0ba5f6e725b3-serving-cert\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.887783 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.887916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-544pm\" (UniqueName: \"kubernetes.io/projected/e938ea40-4c00-45fc-ba25-fa718cf66e06-kube-api-access-544pm\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:21.887916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7txmv\" (UniqueName: \"kubernetes.io/projected/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-kube-api-access-7txmv\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.887916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887861    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxp5g\" (UniqueName: \"kubernetes.io/projected/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-kube-api-access-vxp5g\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.887916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/588349d1-be21-4927-b936-0ba5f6e725b3-tmp\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.888107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887919    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-serving-cert\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.888107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgmqx\" (UniqueName: \"kubernetes.io/projected/7894863b-2223-4519-b749-851b66facc7b-kube-api-access-fgmqx\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.888107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.887978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zq9sf\" (UniqueName: \"kubernetes.io/projected/342114ba-5b4a-43af-9260-78a7e1013d93-kube-api-access-zq9sf\") pod \"volume-data-source-validator-549c6d54cf-clg2w\" (UID: \"342114ba-5b4a-43af-9260-78a7e1013d93\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"
Jul 29 19:30:21.888107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.888045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/588349d1-be21-4927-b936-0ba5f6e725b3-snapshots\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.888282 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.888163    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/588349d1-be21-4927-b936-0ba5f6e725b3-service-ca-bundle\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.888282 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.888249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/588349d1-be21-4927-b936-0ba5f6e725b3-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.888386 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.888333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-trusted-ca\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.888482 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.888440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/588349d1-be21-4927-b936-0ba5f6e725b3-tmp\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.890155 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.890137    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/588349d1-be21-4927-b936-0ba5f6e725b3-serving-cert\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.890364 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.890350    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-serving-cert\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.892561 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.892540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:21.897287 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.897271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqgrz\" (UniqueName: \"kubernetes.io/projected/588349d1-be21-4927-b936-0ba5f6e725b3-kube-api-access-qqgrz\") pod \"insights-operator-5885fdb759-dlrtg\" (UID: \"588349d1-be21-4927-b936-0ba5f6e725b3\") " pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.897674 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.897649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxp5g\" (UniqueName: \"kubernetes.io/projected/8b6c4c13-22b0-41c5-bcb9-ce0a18589113-kube-api-access-vxp5g\") pod \"console-operator-584dcb6b54-jrc7p\" (UID: \"8b6c4c13-22b0-41c5-bcb9-ce0a18589113\") " pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.898063 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.898047    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq9sf\" (UniqueName: \"kubernetes.io/projected/342114ba-5b4a-43af-9260-78a7e1013d93-kube-api-access-zq9sf\") pod \"volume-data-source-validator-549c6d54cf-clg2w\" (UID: \"342114ba-5b4a-43af-9260-78a7e1013d93\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"
Jul 29 19:30:21.954737 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.954701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"
Jul 29 19:30:21.980309 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.980282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:21.988574 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988549    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.988693 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-544pm\" (UniqueName: \"kubernetes.io/projected/e938ea40-4c00-45fc-ba25-fa718cf66e06-kube-api-access-544pm\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:21.988693 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7txmv\" (UniqueName: \"kubernetes.io/projected/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-kube-api-access-7txmv\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.988693 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988665    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgmqx\" (UniqueName: \"kubernetes.io/projected/7894863b-2223-4519-b749-851b66facc7b-kube-api-access-fgmqx\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.988693 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.988689    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/efb86fd8-4782-4e7d-b732-52987e85c001-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.988750    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls podName:efb86fd8-4782-4e7d-b732-52987e85c001 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:22.488727041 +0000 UTC m=+44.366581347 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rftbz" (UID: "efb86fd8-4782-4e7d-b732-52987e85c001") : secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.988788    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7894863b-2223-4519-b749-851b66facc7b-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.988822    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls podName:e938ea40-4c00-45fc-ba25-fa718cf66e06 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:22.488811816 +0000 UTC m=+44.366666122 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls") pod "cluster-samples-operator-6748489656-mwfzv" (UID: "e938ea40-4c00-45fc-ba25-fa718cf66e06") : secret "samples-operator-tls" not found
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988858    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8b64ee3f-af74-4309-a2c2-e3071335f57d-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988918    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6sp8\" (UniqueName: \"kubernetes.io/projected/8b64ee3f-af74-4309-a2c2-e3071335f57d-kube-api-access-k6sp8\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:21.989011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.988960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-default-certificate\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.989517 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.989267    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:22.488980582 +0000 UTC m=+44.366834892 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : configmap references non-existent config key: service-ca.crt
Jul 29 19:30:21.989517 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.989334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj7w4\" (UniqueName: \"kubernetes.io/projected/efb86fd8-4782-4e7d-b732-52987e85c001-kube-api-access-zj7w4\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.989517 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.989335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7894863b-2223-4519-b749-851b66facc7b-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.989517 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.989449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.989517 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.989491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-stats-auth\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.989721 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.989528    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 19:30:21.989721 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.989523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8b64ee3f-af74-4309-a2c2-e3071335f57d-config\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:21.989721 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:21.989581    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:22.48956389 +0000 UTC m=+44.367418196 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : secret "router-metrics-certs-default" not found
Jul 29 19:30:21.989721 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.989630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7894863b-2223-4519-b749-851b66facc7b-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.990120 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.990094    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/efb86fd8-4782-4e7d-b732-52987e85c001-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:21.991728 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.991708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:21.992658 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.992635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-default-certificate\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.992744 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.992646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7894863b-2223-4519-b749-851b66facc7b-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.992782 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.992762    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-dlrtg"
Jul 29 19:30:21.992819 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.992798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-stats-auth\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.998552 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.998522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-544pm\" (UniqueName: \"kubernetes.io/projected/e938ea40-4c00-45fc-ba25-fa718cf66e06-kube-api-access-544pm\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:21.998920 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.998904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgmqx\" (UniqueName: \"kubernetes.io/projected/7894863b-2223-4519-b749-851b66facc7b-kube-api-access-fgmqx\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns\" (UID: \"7894863b-2223-4519-b749-851b66facc7b\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:21.999096 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.999076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7txmv\" (UniqueName: \"kubernetes.io/projected/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-kube-api-access-7txmv\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:21.999826 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:21.999807    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj7w4\" (UniqueName: \"kubernetes.io/projected/efb86fd8-4782-4e7d-b732-52987e85c001-kube-api-access-zj7w4\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:22.026491 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.026181    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64"]
Jul 29 19:30:22.029301 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:22.029275    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9203ac2b_f4c8_453a_b5d5_aff9e4b5d125.slice/crio-dbd4e1f116d3961bcce73495c230099ed95f6e851baa9829909f4b108f68d845 WatchSource:0}: Error finding container dbd4e1f116d3961bcce73495c230099ed95f6e851baa9829909f4b108f68d845: Status 404 returned error can't find the container with id dbd4e1f116d3961bcce73495c230099ed95f6e851baa9829909f4b108f68d845
Jul 29 19:30:22.099488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.097450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"
Jul 29 19:30:22.099488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.098195    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8b64ee3f-af74-4309-a2c2-e3071335f57d-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.099488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.098229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6sp8\" (UniqueName: \"kubernetes.io/projected/8b64ee3f-af74-4309-a2c2-e3071335f57d-kube-api-access-k6sp8\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.099488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.098293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8b64ee3f-af74-4309-a2c2-e3071335f57d-config\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.100202 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.100133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8b64ee3f-af74-4309-a2c2-e3071335f57d-config\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.104574 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.104522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8b64ee3f-af74-4309-a2c2-e3071335f57d-serving-cert\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.111482 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.111440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6sp8\" (UniqueName: \"kubernetes.io/projected/8b64ee3f-af74-4309-a2c2-e3071335f57d-kube-api-access-k6sp8\") pod \"service-ca-operator-5d6c75f84f-c4pmh\" (UID: \"8b64ee3f-af74-4309-a2c2-e3071335f57d\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.117717 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.117690    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"
Jul 29 19:30:22.145467 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.145389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w"]
Jul 29 19:30:22.152793 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:22.152766    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod342114ba_5b4a_43af_9260_78a7e1013d93.slice/crio-f28abd443c606eb6cd19564178b214424f54f1be4b32d6e710ccb038d3521afa WatchSource:0}: Error finding container f28abd443c606eb6cd19564178b214424f54f1be4b32d6e710ccb038d3521afa: Status 404 returned error can't find the container with id f28abd443c606eb6cd19564178b214424f54f1be4b32d6e710ccb038d3521afa
Jul 29 19:30:22.189970 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.189937    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-jrc7p"]
Jul 29 19:30:22.192454 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:22.192407    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b6c4c13_22b0_41c5_bcb9_ce0a18589113.slice/crio-591bb97791942b7c2f357f2bae4a4f83636842ee54a10977159ec71ce310af82 WatchSource:0}: Error finding container 591bb97791942b7c2f357f2bae4a4f83636842ee54a10977159ec71ce310af82: Status 404 returned error can't find the container with id 591bb97791942b7c2f357f2bae4a4f83636842ee54a10977159ec71ce310af82
Jul 29 19:30:22.217484 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.217372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-dlrtg"]
Jul 29 19:30:22.221206 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:22.221179    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588349d1_be21_4927_b936_0ba5f6e725b3.slice/crio-e1623129aa391af424c95806f13ee5e35a0b55eee961312e179630a2dc0e916d WatchSource:0}: Error finding container e1623129aa391af424c95806f13ee5e35a0b55eee961312e179630a2dc0e916d: Status 404 returned error can't find the container with id e1623129aa391af424c95806f13ee5e35a0b55eee961312e179630a2dc0e916d
Jul 29 19:30:22.273632 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.273600    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns"]
Jul 29 19:30:22.276516 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:22.276491    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7894863b_2223_4519_b749_851b66facc7b.slice/crio-a43f5c694cfd4cba7f897378dbf7705cd16dd32d8c4d033270325cb2faf06add WatchSource:0}: Error finding container a43f5c694cfd4cba7f897378dbf7705cd16dd32d8c4d033270325cb2faf06add: Status 404 returned error can't find the container with id a43f5c694cfd4cba7f897378dbf7705cd16dd32d8c4d033270325cb2faf06add
Jul 29 19:30:22.302849 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.302785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh"]
Jul 29 19:30:22.305559 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:22.305532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b64ee3f_af74_4309_a2c2_e3071335f57d.slice/crio-0b2ecbc9464b0e18cf7af423ef1861030ff97e16ae458c541e2d29d9fe357ae6 WatchSource:0}: Error finding container 0b2ecbc9464b0e18cf7af423ef1861030ff97e16ae458c541e2d29d9fe357ae6: Status 404 returned error can't find the container with id 0b2ecbc9464b0e18cf7af423ef1861030ff97e16ae458c541e2d29d9fe357ae6
Jul 29 19:30:22.503686 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.503638    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:22.503860 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.503722    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:22.503860 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.503781    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:22.503860 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503819    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:23.503800069 +0000 UTC m=+45.381654389 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : configmap references non-existent config key: service-ca.crt
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503881    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:22.503883    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503902    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503918    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls podName:efb86fd8-4782-4e7d-b732-52987e85c001 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:23.503907226 +0000 UTC m=+45.381761546 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rftbz" (UID: "efb86fd8-4782-4e7d-b732-52987e85c001") : secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503955    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503961    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:23.50394854 +0000 UTC m=+45.381802846 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : secret "router-metrics-certs-default" not found
Jul 29 19:30:22.504040 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:22.503987    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls podName:e938ea40-4c00-45fc-ba25-fa718cf66e06 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:23.503976949 +0000 UTC m=+45.381831256 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls") pod "cluster-samples-operator-6748489656-mwfzv" (UID: "e938ea40-4c00-45fc-ba25-fa718cf66e06") : secret "samples-operator-tls" not found
Jul 29 19:30:23.014442 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.014398    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-dlrtg" event={"ID":"588349d1-be21-4927-b936-0ba5f6e725b3","Type":"ContainerStarted","Data":"e1623129aa391af424c95806f13ee5e35a0b55eee961312e179630a2dc0e916d"}
Jul 29 19:30:23.015731 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.015700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh" event={"ID":"8b64ee3f-af74-4309-a2c2-e3071335f57d","Type":"ContainerStarted","Data":"0b2ecbc9464b0e18cf7af423ef1861030ff97e16ae458c541e2d29d9fe357ae6"}
Jul 29 19:30:23.017110 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.017078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns" event={"ID":"7894863b-2223-4519-b749-851b66facc7b","Type":"ContainerStarted","Data":"a43f5c694cfd4cba7f897378dbf7705cd16dd32d8c4d033270325cb2faf06add"}
Jul 29 19:30:23.018693 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.018663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" event={"ID":"8b6c4c13-22b0-41c5-bcb9-ce0a18589113","Type":"ContainerStarted","Data":"591bb97791942b7c2f357f2bae4a4f83636842ee54a10977159ec71ce310af82"}
Jul 29 19:30:23.020093 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.020028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w" event={"ID":"342114ba-5b4a-43af-9260-78a7e1013d93","Type":"ContainerStarted","Data":"f28abd443c606eb6cd19564178b214424f54f1be4b32d6e710ccb038d3521afa"}
Jul 29 19:30:23.021325 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.021291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64" event={"ID":"9203ac2b-f4c8-453a-b5d5-aff9e4b5d125","Type":"ContainerStarted","Data":"dbd4e1f116d3961bcce73495c230099ed95f6e851baa9829909f4b108f68d845"}
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.515604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.515687    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.515758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:23.515822    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.515986    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:25.515968059 +0000 UTC m=+47.393822370 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : configmap references non-existent config key: service-ca.crt
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.516428    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.516484    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:25.516467875 +0000 UTC m=+47.394322185 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : secret "router-metrics-certs-default" not found
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.516541    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.516573    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls podName:efb86fd8-4782-4e7d-b732-52987e85c001 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:25.516562632 +0000 UTC m=+47.394416941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rftbz" (UID: "efb86fd8-4782-4e7d-b732-52987e85c001") : secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.516634    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 19:30:23.516751 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:23.516664    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls podName:e938ea40-4c00-45fc-ba25-fa718cf66e06 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:25.516654723 +0000 UTC m=+47.394509031 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls") pod "cluster-samples-operator-6748489656-mwfzv" (UID: "e938ea40-4c00-45fc-ba25-fa718cf66e06") : secret "samples-operator-tls" not found
Jul 29 19:30:25.029635 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.029600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w" event={"ID":"342114ba-5b4a-43af-9260-78a7e1013d93","Type":"ContainerStarted","Data":"3c0a2b58b75a50c6bd2ec8d503de9ef959b57570b3822b9222553e0b9b00c618"}
Jul 29 19:30:25.051870 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.051818    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-clg2w" podStartSLOduration=2.00809706 podStartE2EDuration="4.051802883s" totalImagesPullingTime="2.043705823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:25.050764093 +0000 UTC m=+46.928618422" watchObservedRunningTime="2026-07-29 19:30:25.051802883 +0000 UTC m=+46.929657210"
Jul 29 19:30:25.141466 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.141434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:25.244335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.244297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:25.537843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.537805    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:25.538042 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.537891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:25.538042 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.537952    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:25.538042 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:25.538034    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538073    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538085    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:29.538056489 +0000 UTC m=+51.415910808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : configmap references non-existent config key: service-ca.crt
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538118    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538135    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:29.538118954 +0000 UTC m=+51.415973277 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : secret "router-metrics-certs-default" not found
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538169    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls podName:efb86fd8-4782-4e7d-b732-52987e85c001 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:29.538154228 +0000 UTC m=+51.416008537 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rftbz" (UID: "efb86fd8-4782-4e7d-b732-52987e85c001") : secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538174    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 19:30:25.538225 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:25.538219    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls podName:e938ea40-4c00-45fc-ba25-fa718cf66e06 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:29.53820467 +0000 UTC m=+51.416058982 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls") pod "cluster-samples-operator-6748489656-mwfzv" (UID: "e938ea40-4c00-45fc-ba25-fa718cf66e06") : secret "samples-operator-tls" not found
Jul 29 19:30:26.884123 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:26.884092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:26.944570 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:26.944537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:28.010420 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:28.010388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:28.072100 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:28.072069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:28.150756 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:28.150721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:28.250707 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:28.250678    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:29.043141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.043102    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-dlrtg" event={"ID":"588349d1-be21-4927-b936-0ba5f6e725b3","Type":"ContainerStarted","Data":"79617b526d3fa78387879a31e65b5b477991d9627f3df7b488f62f11a6340caa"}
Jul 29 19:30:29.045108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.045070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh" event={"ID":"8b64ee3f-af74-4309-a2c2-e3071335f57d","Type":"ContainerStarted","Data":"0e2bf63cfbe9193511511925f437fafa58e29558bedcaa9141a857496b6ebba1"}
Jul 29 19:30:29.046799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.046774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns" event={"ID":"7894863b-2223-4519-b749-851b66facc7b","Type":"ContainerStarted","Data":"a78a2970927d149009933b299fceccf6f15abab1e90fadc90d9fe7d513e8db04"}
Jul 29 19:30:29.048706 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.048685    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/0.log"
Jul 29 19:30:29.048813 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.048726    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b6c4c13-22b0-41c5-bcb9-ce0a18589113" containerID="540a2a822290c842d8e561a30680249fc4eaf5981540d98bc12c2f147984a81a" exitCode=255
Jul 29 19:30:29.048874 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.048805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" event={"ID":"8b6c4c13-22b0-41c5-bcb9-ce0a18589113","Type":"ContainerDied","Data":"540a2a822290c842d8e561a30680249fc4eaf5981540d98bc12c2f147984a81a"}
Jul 29 19:30:29.048963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.048946    2580 scope.go:117] "RemoveContainer" containerID="540a2a822290c842d8e561a30680249fc4eaf5981540d98bc12c2f147984a81a"
Jul 29 19:30:29.050846 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.050821    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64" event={"ID":"9203ac2b-f4c8-453a-b5d5-aff9e4b5d125","Type":"ContainerStarted","Data":"a4c6811f296da8e6a7d2978267c556fbb599aee2a8d87c3a0855068343d8eda0"}
Jul 29 19:30:29.071635 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.071578    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-dlrtg" podStartSLOduration=1.7887404930000002 podStartE2EDuration="8.071557135s" totalImagesPullingTime="6.282816642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:29.069356819 +0000 UTC m=+50.947211146" watchObservedRunningTime="2026-07-29 19:30:29.071557135 +0000 UTC m=+50.949411444"
Jul 29 19:30:29.141245 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.141188    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-zkz64" podStartSLOduration=1.665744074 podStartE2EDuration="8.141167391s" totalImagesPullingTime="6.475423317s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:29.114666219 +0000 UTC m=+50.992520548" watchObservedRunningTime="2026-07-29 19:30:29.141167391 +0000 UTC m=+51.019021720"
Jul 29 19:30:29.165947 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.165841    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns" podStartSLOduration=1.937043542 podStartE2EDuration="8.165822309s" totalImagesPullingTime="6.228778767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:29.16471465 +0000 UTC m=+51.042568979" watchObservedRunningTime="2026-07-29 19:30:29.165822309 +0000 UTC m=+51.043676639"
Jul 29 19:30:29.205027 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.203958    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh" podStartSLOduration=1.998158178 podStartE2EDuration="8.203936103s" totalImagesPullingTime="6.205777925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:29.203004464 +0000 UTC m=+51.080858784" watchObservedRunningTime="2026-07-29 19:30:29.203936103 +0000 UTC m=+51.081790433"
Jul 29 19:30:29.578840 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.578792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:29.579038 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.578881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:29.579038 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.578939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:29.579038 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.578978    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:29.579038 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.579007    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:29.579277 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.579074    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls podName:efb86fd8-4782-4e7d-b732-52987e85c001 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:37.579052666 +0000 UTC m=+59.456906975 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rftbz" (UID: "efb86fd8-4782-4e7d-b732-52987e85c001") : secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:29.579277 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.579159    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 19:30:29.579277 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.579213    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:37.579196916 +0000 UTC m=+59.457051231 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : secret "router-metrics-certs-default" not found
Jul 29 19:30:29.579277 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.579272    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 19:30:29.579469 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.579306    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls podName:e938ea40-4c00-45fc-ba25-fa718cf66e06 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:37.57929504 +0000 UTC m=+59.457149346 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls") pod "cluster-samples-operator-6748489656-mwfzv" (UID: "e938ea40-4c00-45fc-ba25-fa718cf66e06") : secret "samples-operator-tls" not found
Jul 29 19:30:29.579469 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.579376    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:37.579365456 +0000 UTC m=+59.457219764 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : configmap references non-existent config key: service-ca.crt
Jul 29 19:30:29.618297 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.618263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:29.734381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.734341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:29.829354 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.829267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:29.882499 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.882458    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:29.882685 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.882609    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 29 19:30:29.882685 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.882627    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-54fcb899b6-zz94f: secret "image-registry-tls" not found
Jul 29 19:30:29.882685 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.882634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:29.882685 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.882680    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls podName:04ff50d9-45ce-4220-80e6-9a042447bde1 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:45.882664928 +0000 UTC m=+67.760519234 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls") pod "image-registry-54fcb899b6-zz94f" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1") : secret "image-registry-tls" not found
Jul 29 19:30:29.882890 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.882756    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 29 19:30:29.882890 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:29.882815    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls podName:f9f66716-3a4b-4276-bef5-7575195acbfc nodeName:}" failed. No retries permitted until 2026-07-29 19:30:45.882798847 +0000 UTC m=+67.760653167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls") pod "dns-default-pclnw" (UID: "f9f66716-3a4b-4276-bef5-7575195acbfc") : secret "dns-default-metrics-tls" not found
Jul 29 19:30:29.965250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:29.965214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.056837 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.056813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:30:30.057324 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.057255    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/0.log"
Jul 29 19:30:30.057324 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.057287    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b6c4c13-22b0-41c5-bcb9-ce0a18589113" containerID="0fdc21ca97bbafb8f611ce1887f6d851a0df5ad87a5e9f5dbe2c300751630d84" exitCode=255
Jul 29 19:30:30.057462 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.057413    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" event={"ID":"8b6c4c13-22b0-41c5-bcb9-ce0a18589113","Type":"ContainerDied","Data":"0fdc21ca97bbafb8f611ce1887f6d851a0df5ad87a5e9f5dbe2c300751630d84"}
Jul 29 19:30:30.057538 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.057516    2580 scope.go:117] "RemoveContainer" containerID="540a2a822290c842d8e561a30680249fc4eaf5981540d98bc12c2f147984a81a"
Jul 29 19:30:30.057675 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.057658    2580 scope.go:117] "RemoveContainer" containerID="0fdc21ca97bbafb8f611ce1887f6d851a0df5ad87a5e9f5dbe2c300751630d84"
Jul 29 19:30:30.057791 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.057733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.057876 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.057852    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-jrc7p_openshift-console-operator(8b6c4c13-22b0-41c5-bcb9-ce0a18589113)\"" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" podUID="8b6c4c13-22b0-41c5-bcb9-ce0a18589113"
Jul 29 19:30:30.138559 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.138512    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vrvjf"]
Jul 29 19:30:30.155464 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.155412    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.168843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.168813    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vrvjf"]
Jul 29 19:30:30.169045 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.168926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.170558 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.170535    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 29 19:30:30.170707 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.170693    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 29 19:30:30.170774 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.170709    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8twzb\""
Jul 29 19:30:30.218521 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.218486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.287448 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.287403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.287631 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.287458    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0db0227f-3080-4a56-8950-bca5e9aeeea8-data-volume\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.287631 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.287488    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mfpw\" (UniqueName: \"kubernetes.io/projected/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-api-access-6mfpw\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.287631 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.287518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0db0227f-3080-4a56-8950-bca5e9aeeea8-crio-socket\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.287754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.287646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389337 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389152    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0db0227f-3080-4a56-8950-bca5e9aeeea8-data-volume\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389337 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mfpw\" (UniqueName: \"kubernetes.io/projected/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-api-access-6mfpw\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389337 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0db0227f-3080-4a56-8950-bca5e9aeeea8-crio-socket\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389337 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.389202    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 19:30:30.389337 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389337 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.389298    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:30.88927552 +0000 UTC m=+52.767129832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : configmap "kube-rbac-proxy" not found
Jul 29 19:30:30.389627 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.389354    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:30.389627 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.389414    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:30.88939904 +0000 UTC m=+52.767253351 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:30.389627 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0db0227f-3080-4a56-8950-bca5e9aeeea8-crio-socket\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.389627 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.389518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0db0227f-3080-4a56-8950-bca5e9aeeea8-data-volume\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.402517 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.402489    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mfpw\" (UniqueName: \"kubernetes.io/projected/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-api-access-6mfpw\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.592546 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.592503    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.684603 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.684520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.791292 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.791262    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.859715 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.859676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:30.892945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.892915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.893120 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:30.892973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:30.893120 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.893082    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 19:30:30.893120 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.893108    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:30.893226 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.893161    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:31.893142324 +0000 UTC m=+53.770996630 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:30.893226 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:30.893181    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:31.893172226 +0000 UTC m=+53.771026532 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : configmap "kube-rbac-proxy" not found
Jul 29 19:30:31.062390 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.062315    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:30:31.062809 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.062699    2580 scope.go:117] "RemoveContainer" containerID="0fdc21ca97bbafb8f611ce1887f6d851a0df5ad87a5e9f5dbe2c300751630d84"
Jul 29 19:30:31.062887 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:31.062867    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-jrc7p_openshift-console-operator(8b6c4c13-22b0-41c5-bcb9-ce0a18589113)\"" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" podUID="8b6c4c13-22b0-41c5-bcb9-ce0a18589113"
Jul 29 19:30:31.157359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.157330    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:31.260070 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.260037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:31.905478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.905432    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:31.905663 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.905506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:31.905663 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:31.905573    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 19:30:31.905663 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:31.905642    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:33.905626678 +0000 UTC m=+55.783480984 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : configmap "kube-rbac-proxy" not found
Jul 29 19:30:31.905778 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:31.905675    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:31.905778 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:31.905733    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:33.90572016 +0000 UTC m=+55.783574466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:31.980775 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.980735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:31.980937 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:31.980816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:32.066095 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:32.066071    2580 scope.go:117] "RemoveContainer" containerID="0fdc21ca97bbafb8f611ce1887f6d851a0df5ad87a5e9f5dbe2c300751630d84"
Jul 29 19:30:32.066455 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:32.066226    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-jrc7p_openshift-console-operator(8b6c4c13-22b0-41c5-bcb9-ce0a18589113)\"" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" podUID="8b6c4c13-22b0-41c5-bcb9-ce0a18589113"
Jul 29 19:30:32.247062 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:32.246973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:32.308318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:32.308287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:32.957135 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:32.957110    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j5vzw_464c7d4e-5049-415c-bf60-3bab6ae5429c/dns-node-resolver/0.log"
Jul 29 19:30:32.960432 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:32.960408    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:33.058774 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:33.058742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:33.773410 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:33.773376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:33.868693 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:33.868663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:33.923871 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:33.923843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:33.924070 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:33.923899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:33.924070 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:33.923974    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 19:30:33.924167 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:33.924076    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:37.924054628 +0000 UTC m=+59.801908945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : configmap "kube-rbac-proxy" not found
Jul 29 19:30:33.924167 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:33.924085    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:33.924167 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:33.924123    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:37.92411262 +0000 UTC m=+59.801966926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:33.956450 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:33.956423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pg9vq_4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4/node-ca/0.log"
Jul 29 19:30:34.757193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:34.757163    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n88rq_e3b0bec5-445e-4fa4-bde1-86becceb24a8/serve-healthcheck-canary/0.log"
Jul 29 19:30:35.561696 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:35.561664    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns_7894863b-2223-4519-b749-851b66facc7b/kube-storage-version-migrator-operator/0.log"
Jul 29 19:30:36.811159 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:36.811121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:36.977110 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:36.977080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:37.477821 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.477785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:37.576804 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.576774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:37.656573 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.656536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:37.656750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.656623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:37.656750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.656693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:37.656750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.656747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:37.656897 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656697    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:37.656897 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656773    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 29 19:30:37.656897 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656793    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:53.656774331 +0000 UTC m=+75.534628637 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : configmap references non-existent config key: service-ca.crt
Jul 29 19:30:37.656897 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656841    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls podName:e938ea40-4c00-45fc-ba25-fa718cf66e06 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:53.6568226 +0000 UTC m=+75.534676907 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls") pod "cluster-samples-operator-6748489656-mwfzv" (UID: "e938ea40-4c00-45fc-ba25-fa718cf66e06") : secret "samples-operator-tls" not found
Jul 29 19:30:37.656897 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656861    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls podName:efb86fd8-4782-4e7d-b732-52987e85c001 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:53.656851695 +0000 UTC m=+75.534706005 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-rftbz" (UID: "efb86fd8-4782-4e7d-b732-52987e85c001") : secret "cluster-monitoring-operator-tls" not found
Jul 29 19:30:37.656897 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656871    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 29 19:30:37.657190 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.656927    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs podName:c83babd6-f73c-4bdf-8ee2-bf8ade50ae73 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:53.656917243 +0000 UTC m=+75.534771549 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs") pod "router-default-66df6dd487-2zjcx" (UID: "c83babd6-f73c-4bdf-8ee2-bf8ade50ae73") : secret "router-metrics-certs-default" not found
Jul 29 19:30:37.680768 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.680740    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:37.834884 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.834806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:37.959092 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.959052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:37.959295 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:37.959115    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:37.959295 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.959193    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 29 19:30:37.959295 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.959218    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:37.959295 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.959270    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:45.959249904 +0000 UTC m=+67.837104210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : configmap "kube-rbac-proxy" not found
Jul 29 19:30:37.959295 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:37.959288    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:45.959280319 +0000 UTC m=+67.837134627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:38.880768 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:38.880731    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:39.040734 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:39.040698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:39.218014 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:39.217917    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:39.614954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:39.614875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:39.978823 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:39.978793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t6vcr"
Jul 29 19:30:40.133826 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:40.133797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:40.427914 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:40.427882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:40.641459 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:40.641431    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:40.699768 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:40.699690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:40.930140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:40.930113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:41.034726 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:41.034643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:41.228281 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:41.228250    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:41.615008 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:41.614965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:41.668955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:41.668922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:42.227870 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:42.227844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:42.828962 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:42.828936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:43.016726 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.016691    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:43.093252 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.093171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:43.220956 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.220923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:43.279485 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.279453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:43.411062 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.411033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:30:43.411237 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.411077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:43.412635 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.412619    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 29 19:30:43.412722 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.412659    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 29 19:30:43.422045 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:43.422024    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 29 19:30:43.422128 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:43.422093    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs podName:25377374-58d1-4eab-be9b-6a35696cc589 nodeName:}" failed. No retries permitted until 2026-07-29 19:31:47.422076781 +0000 UTC m=+129.299931087 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs") pod "network-metrics-daemon-rx6zl" (UID: "25377374-58d1-4eab-be9b-6a35696cc589") : secret "metrics-daemon-secret" not found
Jul 29 19:30:43.424346 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.424327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/6b8bf68c-0fd8-442c-af0b-90e8fbe11f84-original-pull-secret\") pod \"global-pull-secret-syncer-ffjxb\" (UID: \"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84\") " pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:43.511868 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.511831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:43.514398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.514380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8whn\" (UniqueName: \"kubernetes.io/projected/7264fa1c-f617-43fd-a8d7-cfe79f7fc782-kube-api-access-q8whn\") pod \"network-check-target-bmrsw\" (UID: \"7264fa1c-f617-43fd-a8d7-cfe79f7fc782\") " pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:43.621626 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.621601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ffjxb"
Jul 29 19:30:43.638064 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.638040    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qwcp5\""
Jul 29 19:30:43.647506 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.647485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:43.656124 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.656023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:43.759316 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.759182    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ffjxb"]
Jul 29 19:30:43.762184 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:43.762155    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8bf68c_0fd8_442c_af0b_90e8fbe11f84.slice/crio-0ae764bc3ed7a5280f71ba44ac6e7e2a2ce94d08079b4a38c00827335815bff9 WatchSource:0}: Error finding container 0ae764bc3ed7a5280f71ba44ac6e7e2a2ce94d08079b4a38c00827335815bff9: Status 404 returned error can't find the container with id 0ae764bc3ed7a5280f71ba44ac6e7e2a2ce94d08079b4a38c00827335815bff9
Jul 29 19:30:43.778390 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:43.778366    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bmrsw"]
Jul 29 19:30:43.780988 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:43.780966    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7264fa1c_f617_43fd_a8d7_cfe79f7fc782.slice/crio-c027bcc4f8fac199e77ac51dcbb7339416c708fe1852c11d94c49031340c5d24 WatchSource:0}: Error finding container c027bcc4f8fac199e77ac51dcbb7339416c708fe1852c11d94c49031340c5d24: Status 404 returned error can't find the container with id c027bcc4f8fac199e77ac51dcbb7339416c708fe1852c11d94c49031340c5d24
Jul 29 19:30:44.103606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.103514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffjxb" event={"ID":"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84","Type":"ContainerStarted","Data":"0ae764bc3ed7a5280f71ba44ac6e7e2a2ce94d08079b4a38c00827335815bff9"}
Jul 29 19:30:44.104858 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.104828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bmrsw" event={"ID":"7264fa1c-f617-43fd-a8d7-cfe79f7fc782","Type":"ContainerStarted","Data":"f076f0522d188cb9ffa5a374c7d1be522ca7918087cf194d56923f3f4e5fa911"}
Jul 29 19:30:44.105024 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.104865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bmrsw" event={"ID":"7264fa1c-f617-43fd-a8d7-cfe79f7fc782","Type":"ContainerStarted","Data":"c027bcc4f8fac199e77ac51dcbb7339416c708fe1852c11d94c49031340c5d24"}
Jul 29 19:30:44.105024 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.104893    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:30:44.126728 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.124545    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bmrsw" podStartSLOduration=66.124516601 podStartE2EDuration="1m6.124516601s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:44.122305743 +0000 UTC m=+66.000160086" watchObservedRunningTime="2026-07-29 19:30:44.124516601 +0000 UTC m=+66.002370931"
Jul 29 19:30:44.217228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.217189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:44.424876 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.424800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:44.710800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:44.710730    2580 scope.go:117] "RemoveContainer" containerID="0fdc21ca97bbafb8f611ce1887f6d851a0df5ad87a5e9f5dbe2c300751630d84"
Jul 29 19:30:45.016593 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.016512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:45.110605 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.110576    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:30:45.110792 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.110714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" event={"ID":"8b6c4c13-22b0-41c5-bcb9-ce0a18589113","Type":"ContainerStarted","Data":"04dde9a8f242dc1b69f1843ba392610d768e205b8b5cf8fb7c0767a848416636"}
Jul 29 19:30:45.111047 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.111024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:45.128892 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.128829    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p" podStartSLOduration=17.816030749 podStartE2EDuration="24.128802463s" totalImagesPullingTime="6.312771714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:45.127875451 +0000 UTC m=+67.005729784" watchObservedRunningTime="2026-07-29 19:30:45.128802463 +0000 UTC m=+67.006656791"
Jul 29 19:30:45.417984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.417950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:45.469467 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.469438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:45.485095 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.485066    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-jrc7p"
Jul 29 19:30:45.678949 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.678873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:45.774352 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.774322    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:45.874984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.874932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:45.935126 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.935048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:45.935126 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.935104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:45.937699 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.937675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9f66716-3a4b-4276-bef5-7575195acbfc-metrics-tls\") pod \"dns-default-pclnw\" (UID: \"f9f66716-3a4b-4276-bef5-7575195acbfc\") " pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:45.937842 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.937763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"image-registry-54fcb899b6-zz94f\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") " pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:45.974790 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:45.974751    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:46.036206 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.036156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:46.036411 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.036263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:46.036411 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:46.036385    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:46.036536 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:46.036461    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls podName:0db0227f-3080-4a56-8950-bca5e9aeeea8 nodeName:}" failed. No retries permitted until 2026-07-29 19:31:02.036440266 +0000 UTC m=+83.914294575 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-vrvjf" (UID: "0db0227f-3080-4a56-8950-bca5e9aeeea8") : secret "insights-runtime-extractor-tls" not found
Jul 29 19:30:46.036769 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.036749    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0db0227f-3080-4a56-8950-bca5e9aeeea8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:30:46.077407 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.077371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:46.094986 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.094955    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-95c74\""
Jul 29 19:30:46.104310 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.104277    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:46.118743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.118718    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8z4sg\""
Jul 29 19:30:46.127342 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.127324    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:46.288016 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.287957    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-54fcb899b6-zz94f"]
Jul 29 19:30:46.291649 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:46.291614    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ff50d9_45ce_4220_80e6_9a042447bde1.slice/crio-b87c46d18e21c4f8489268e60175e6b8758cca3a702312c31c6e254e290c74f0 WatchSource:0}: Error finding container b87c46d18e21c4f8489268e60175e6b8758cca3a702312c31c6e254e290c74f0: Status 404 returned error can't find the container with id b87c46d18e21c4f8489268e60175e6b8758cca3a702312c31c6e254e290c74f0
Jul 29 19:30:46.312291 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.312261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pclnw"]
Jul 29 19:30:46.316010 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:46.315966    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9f66716_3a4b_4276_bef5_7575195acbfc.slice/crio-23fad06a4f60193dbfdfde3ac377c4b0c26b7637419fed98951f79ed07768586 WatchSource:0}: Error finding container 23fad06a4f60193dbfdfde3ac377c4b0c26b7637419fed98951f79ed07768586: Status 404 returned error can't find the container with id 23fad06a4f60193dbfdfde3ac377c4b0c26b7637419fed98951f79ed07768586
Jul 29 19:30:46.347225 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.347179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:46.449527 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.449449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:46.508750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.508721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:46.563483 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:46.563454    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:47.021331 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.021294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:47.124519 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.124477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pclnw" event={"ID":"f9f66716-3a4b-4276-bef5-7575195acbfc","Type":"ContainerStarted","Data":"23fad06a4f60193dbfdfde3ac377c4b0c26b7637419fed98951f79ed07768586"}
Jul 29 19:30:47.126264 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.126222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f" event={"ID":"04ff50d9-45ce-4220-80e6-9a042447bde1","Type":"ContainerStarted","Data":"58694955d1c2c295ea421ba6f9226519927f3ae350864b17d0974b91b5a05eed"}
Jul 29 19:30:47.126407 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.126286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f" event={"ID":"04ff50d9-45ce-4220-80e6-9a042447bde1","Type":"ContainerStarted","Data":"b87c46d18e21c4f8489268e60175e6b8758cca3a702312c31c6e254e290c74f0"}
Jul 29 19:30:47.149446 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.149364    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f" podStartSLOduration=69.149343444 podStartE2EDuration="1m9.149343444s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:47.149254387 +0000 UTC m=+69.027108719" watchObservedRunningTime="2026-07-29 19:30:47.149343444 +0000 UTC m=+69.027197774"
Jul 29 19:30:47.418336 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.418301    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:47.573681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.573642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:47.674523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.674430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:47.775883 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:47.775856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:48.130699 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:48.130665    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:30:48.430110 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:48.430077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:48.492226 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:48.492193    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:49.137054 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:49.137019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ffjxb" event={"ID":"6b8bf68c-0fd8-442c-af0b-90e8fbe11f84","Type":"ContainerStarted","Data":"1ca0a4ebaf8ff990b12852d63940072c258af8cb367925ee0f7d63db648650fb"}
Jul 29 19:30:50.140857 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:50.140817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pclnw" event={"ID":"f9f66716-3a4b-4276-bef5-7575195acbfc","Type":"ContainerStarted","Data":"ec1ce94fa060f794488d8107230ca6a99584e0472651f206ba8a0c9ce54b85d9"}
Jul 29 19:30:50.141325 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:50.140875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pclnw" event={"ID":"f9f66716-3a4b-4276-bef5-7575195acbfc","Type":"ContainerStarted","Data":"39090dc6f037cc27c6b0a485eb4f7216fce9e2eac5c5eebdb89f9c0cb5466e5e"}
Jul 29 19:30:50.161684 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:50.161627    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ffjxb" podStartSLOduration=67.856299 podStartE2EDuration="1m12.161610612s" totalImagesPullingTime="4.305311612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:49.153159767 +0000 UTC m=+71.031014095" watchObservedRunningTime="2026-07-29 19:30:50.161610612 +0000 UTC m=+72.039464939"
Jul 29 19:30:50.162858 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:50.162824    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-pclnw" podStartSLOduration=34.301698995 podStartE2EDuration="37.16281338s" totalImagesPullingTime="2.861114385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:50.161572087 +0000 UTC m=+72.039426418" watchObservedRunningTime="2026-07-29 19:30:50.16281338 +0000 UTC m=+72.040667708"
Jul 29 19:30:51.144233 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:51.144198    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pclnw"
Jul 29 19:30:53.703709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.703672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:53.704136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.703729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:53.704136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.703767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:53.704136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.703804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:53.704528 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.704464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-service-ca-bundle\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:53.706276 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.706248    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/efb86fd8-4782-4e7d-b732-52987e85c001-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-rftbz\" (UID: \"efb86fd8-4782-4e7d-b732-52987e85c001\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:53.706362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.706288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/e938ea40-4c00-45fc-ba25-fa718cf66e06-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-mwfzv\" (UID: \"e938ea40-4c00-45fc-ba25-fa718cf66e06\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:53.706362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.706290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c83babd6-f73c-4bdf-8ee2-bf8ade50ae73-metrics-certs\") pod \"router-default-66df6dd487-2zjcx\" (UID: \"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73\") " pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:53.818799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.818765    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-56md8\""
Jul 29 19:30:53.827477 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.827456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"
Jul 29 19:30:53.842288 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.842267    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-b9qrq\""
Jul 29 19:30:53.850713 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.850683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:53.857052 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.857028    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-6d2fs\""
Jul 29 19:30:53.865500 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.865471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"
Jul 29 19:30:53.978711 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:53.978685    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv"]
Jul 29 19:30:54.004558 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.004507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66df6dd487-2zjcx"]
Jul 29 19:30:54.007694 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:54.007652    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc83babd6_f73c_4bdf_8ee2_bf8ade50ae73.slice/crio-cc9129e2f480c478e3c5f19bcf18c5d812bd465bf1a27920a3db10ccf905907f WatchSource:0}: Error finding container cc9129e2f480c478e3c5f19bcf18c5d812bd465bf1a27920a3db10ccf905907f: Status 404 returned error can't find the container with id cc9129e2f480c478e3c5f19bcf18c5d812bd465bf1a27920a3db10ccf905907f
Jul 29 19:30:54.026245 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.026218    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz"]
Jul 29 19:30:54.030255 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:54.030229    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefb86fd8_4782_4e7d_b732_52987e85c001.slice/crio-992acd167fc2497d30cb8c7e29ac28cd43a22605a1690d8da245a2fe3f532626 WatchSource:0}: Error finding container 992acd167fc2497d30cb8c7e29ac28cd43a22605a1690d8da245a2fe3f532626: Status 404 returned error can't find the container with id 992acd167fc2497d30cb8c7e29ac28cd43a22605a1690d8da245a2fe3f532626
Jul 29 19:30:54.156326 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.156287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz" event={"ID":"efb86fd8-4782-4e7d-b732-52987e85c001","Type":"ContainerStarted","Data":"992acd167fc2497d30cb8c7e29ac28cd43a22605a1690d8da245a2fe3f532626"}
Jul 29 19:30:54.157762 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.157736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66df6dd487-2zjcx" event={"ID":"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73","Type":"ContainerStarted","Data":"194a883a9a9a9b1f807b20a6c09cdbadb6a7874d055a2efd763f7612319be1a8"}
Jul 29 19:30:54.157906 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.157786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66df6dd487-2zjcx" event={"ID":"c83babd6-f73c-4bdf-8ee2-bf8ade50ae73","Type":"ContainerStarted","Data":"cc9129e2f480c478e3c5f19bcf18c5d812bd465bf1a27920a3db10ccf905907f"}
Jul 29 19:30:54.158961 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.158936    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv" event={"ID":"e938ea40-4c00-45fc-ba25-fa718cf66e06","Type":"ContainerStarted","Data":"222c15786c05f45baeb28e394af387f43cb39164a0e37ffc7e25c1995f75434f"}
Jul 29 19:30:54.176096 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.176037    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-66df6dd487-2zjcx" podStartSLOduration=33.176019963 podStartE2EDuration="33.176019963s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:54.175428734 +0000 UTC m=+76.053283062" watchObservedRunningTime="2026-07-29 19:30:54.176019963 +0000 UTC m=+76.053874283"
Jul 29 19:30:54.186689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.186657    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:54.851904 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.851865    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:54.855161 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:54.855134    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:55.163639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:55.163609    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:55.164954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:55.164930    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-66df6dd487-2zjcx"
Jul 29 19:30:55.505512 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:55.505472    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:55.592127 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:55.592089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:55.691329 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:55.691294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:55.790795 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:55.790720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:56.474151 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.474115    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:56.581167 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.581079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:56.679555 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.679526    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:56.869873 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.869792    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"]
Jul 29 19:30:56.871860 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.871843    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:56.873527 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.873511    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Jul 29 19:30:56.873600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.873540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-db2td\""
Jul 29 19:30:56.882121 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.882097    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"]
Jul 29 19:30:56.932942 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:56.932906    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/041a18b2-f783-458b-adbf-4e8abef2fd61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qd92r\" (UID: \"041a18b2-f783-458b-adbf-4e8abef2fd61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:57.033487 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.033448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/041a18b2-f783-458b-adbf-4e8abef2fd61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qd92r\" (UID: \"041a18b2-f783-458b-adbf-4e8abef2fd61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:57.033656 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:57.033571    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 19:30:57.033656 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:30:57.033625    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/041a18b2-f783-458b-adbf-4e8abef2fd61-tls-certificates podName:041a18b2-f783-458b-adbf-4e8abef2fd61 nodeName:}" failed. No retries permitted until 2026-07-29 19:30:57.533611348 +0000 UTC m=+79.411465654 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/041a18b2-f783-458b-adbf-4e8abef2fd61-tls-certificates") pod "prometheus-operator-admission-webhook-7f8d78f54b-qd92r" (UID: "041a18b2-f783-458b-adbf-4e8abef2fd61") : secret "prometheus-operator-admission-webhook-tls" not found
Jul 29 19:30:57.172203 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.172166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz" event={"ID":"efb86fd8-4782-4e7d-b732-52987e85c001","Type":"ContainerStarted","Data":"9700a686c45d25bebe8cbe6673f134a1a5bdf0d4021cbbd5b669d3fb22920f4a"}
Jul 29 19:30:57.174032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.173982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv" event={"ID":"e938ea40-4c00-45fc-ba25-fa718cf66e06","Type":"ContainerStarted","Data":"b4aa5421a0fab078f81f55d025788ad438802c75b2d17984de153e285f7e0e46"}
Jul 29 19:30:57.174177 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.174058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv" event={"ID":"e938ea40-4c00-45fc-ba25-fa718cf66e06","Type":"ContainerStarted","Data":"1871fc7c7a8a5c95327fe6eee06c30d689a03de3348bdd0aac66754276b21f81"}
Jul 29 19:30:57.189250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.189202    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-rftbz" podStartSLOduration=33.942440929 podStartE2EDuration="36.189186792s" totalImagesPullingTime="2.246745863s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:57.187737662 +0000 UTC m=+79.065591990" watchObservedRunningTime="2026-07-29 19:30:57.189186792 +0000 UTC m=+79.067041120"
Jul 29 19:30:57.203212 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.203168    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-mwfzv" podStartSLOduration=33.960301393 podStartE2EDuration="36.203153299s" totalImagesPullingTime="2.242851906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:57.20226066 +0000 UTC m=+79.080114988" watchObservedRunningTime="2026-07-29 19:30:57.203153299 +0000 UTC m=+79.081007640"
Jul 29 19:30:57.347021 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.346968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:57.404069 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.404030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:57.538485 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.538398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/041a18b2-f783-458b-adbf-4e8abef2fd61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qd92r\" (UID: \"041a18b2-f783-458b-adbf-4e8abef2fd61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:57.540956 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.540936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/041a18b2-f783-458b-adbf-4e8abef2fd61-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-qd92r\" (UID: \"041a18b2-f783-458b-adbf-4e8abef2fd61\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:57.780944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.780895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:57.907848 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:57.907823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"]
Jul 29 19:30:57.910545 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:30:57.910516    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod041a18b2_f783_458b_adbf_4e8abef2fd61.slice/crio-6cdf736e92f4c125235413679a56761052f1da6e84a2fd0a7f51109036225ede WatchSource:0}: Error finding container 6cdf736e92f4c125235413679a56761052f1da6e84a2fd0a7f51109036225ede: Status 404 returned error can't find the container with id 6cdf736e92f4c125235413679a56761052f1da6e84a2fd0a7f51109036225ede
Jul 29 19:30:58.178329 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:58.178295    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r" event={"ID":"041a18b2-f783-458b-adbf-4e8abef2fd61","Type":"ContainerStarted","Data":"6cdf736e92f4c125235413679a56761052f1da6e84a2fd0a7f51109036225ede"}
Jul 29 19:30:58.189700 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:58.189672    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:58.251950 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:58.251895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:59.184488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.184457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r" event={"ID":"041a18b2-f783-458b-adbf-4e8abef2fd61","Type":"ContainerStarted","Data":"5e8d8b886067758a9032d8342cd8d42e342a0fe8596eecada6b0f7acf4cc4766"}
Jul 29 19:30:59.184932 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.184595    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:59.189088 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.189061    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r"
Jul 29 19:30:59.199601 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.199558    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-qd92r" podStartSLOduration=2.052297744 podStartE2EDuration="3.19954265s" totalImagesPullingTime="1.147244906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:30:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:30:59.198486329 +0000 UTC m=+81.076340659" watchObservedRunningTime="2026-07-29 19:30:59.19954265 +0000 UTC m=+81.077396977"
Jul 29 19:30:59.699538 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.699419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:59.800469 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.800425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:59.900201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.900169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:30:59.980812 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:30:59.980727    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:00.038316 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:00.038282    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:01.150417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.150381    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pclnw"
Jul 29 19:31:01.793068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.793035    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5d576c7d69-q2jsg"]
Jul 29 19:31:01.824310 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.824281    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d576c7d69-q2jsg"]
Jul 29 19:31:01.824463 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.824381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.826474 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826446    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 29 19:31:01.826614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fwg7n\""
Jul 29 19:31:01.826614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826529    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 29 19:31:01.826614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826586    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 29 19:31:01.826614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826606    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 29 19:31:01.826907 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826889    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 29 19:31:01.827019 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826965    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 29 19:31:01.827019 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.826971    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 29 19:31:01.877361 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.877335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-service-ca\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.877511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.877388    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-oauth-serving-cert\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.877511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.877412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-config\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.877511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.877429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjhcs\" (UniqueName: \"kubernetes.io/projected/2bfba303-d00b-417a-bd89-bf0f1047ad35-kube-api-access-jjhcs\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.877511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.877485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-oauth-config\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.877640 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.877513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-serving-cert\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.978682 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.978641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-service-ca\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.978881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.978711    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-oauth-serving-cert\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.978881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.978742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-config\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.978881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.978765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjhcs\" (UniqueName: \"kubernetes.io/projected/2bfba303-d00b-417a-bd89-bf0f1047ad35-kube-api-access-jjhcs\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.979079 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.978909    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-oauth-config\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.979079 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.978976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-serving-cert\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.979528 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.979499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-oauth-serving-cert\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.979806 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.979524    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-service-ca\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.979806 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.979562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-config\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.981463 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.981442    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-oauth-config\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.981849 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.981825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-serving-cert\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:01.988406 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:01.988370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjhcs\" (UniqueName: \"kubernetes.io/projected/2bfba303-d00b-417a-bd89-bf0f1047ad35-kube-api-access-jjhcs\") pod \"console-5d576c7d69-q2jsg\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") " pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:02.080310 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.080227    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:31:02.082657 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.082627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0db0227f-3080-4a56-8950-bca5e9aeeea8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vrvjf\" (UID: \"0db0227f-3080-4a56-8950-bca5e9aeeea8\") " pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:31:02.133460 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.133429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:02.280036 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.279989    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8twzb\""
Jul 29 19:31:02.289442 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.289416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vrvjf"
Jul 29 19:31:02.295769 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.295749    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5d576c7d69-q2jsg"]
Jul 29 19:31:02.307861 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:31:02.307835    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfba303_d00b_417a_bd89_bf0f1047ad35.slice/crio-a263449884935bbd8a77447bbc93c9a5294e6c53059ae0fb3b77964cfca09e91 WatchSource:0}: Error finding container a263449884935bbd8a77447bbc93c9a5294e6c53059ae0fb3b77964cfca09e91: Status 404 returned error can't find the container with id a263449884935bbd8a77447bbc93c9a5294e6c53059ae0fb3b77964cfca09e91
Jul 29 19:31:02.439759 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:02.439734    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vrvjf"]
Jul 29 19:31:02.441804 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:31:02.441778    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db0227f_3080_4a56_8950_bca5e9aeeea8.slice/crio-a33c4e9a18f39770d4d8fcaa3b85653977c4d4b30d1c2d704cdc6da218142cd7 WatchSource:0}: Error finding container a33c4e9a18f39770d4d8fcaa3b85653977c4d4b30d1c2d704cdc6da218142cd7: Status 404 returned error can't find the container with id a33c4e9a18f39770d4d8fcaa3b85653977c4d4b30d1c2d704cdc6da218142cd7
Jul 29 19:31:03.201148 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:03.201054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vrvjf" event={"ID":"0db0227f-3080-4a56-8950-bca5e9aeeea8","Type":"ContainerStarted","Data":"a33c4e9a18f39770d4d8fcaa3b85653977c4d4b30d1c2d704cdc6da218142cd7"}
Jul 29 19:31:03.202946 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:03.202919    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d576c7d69-q2jsg" event={"ID":"2bfba303-d00b-417a-bd89-bf0f1047ad35","Type":"ContainerStarted","Data":"a263449884935bbd8a77447bbc93c9a5294e6c53059ae0fb3b77964cfca09e91"}
Jul 29 19:31:03.798961 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:03.798926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:03.897925 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:03.897895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:04.000103 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:04.000071    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:04.171045 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:04.170988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:04.263579 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:04.263541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:04.383473 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:04.383435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:04.762312 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:04.762280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:04.864986 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:04.864922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:05.190841 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:05.190771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:05.591078 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:05.590972    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:05.691495 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:05.691435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:06.434516 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:06.434482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:06.515681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:06.515646    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.002178 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.002144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.111391 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.111359    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.205292 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.205253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.309743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.309661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.399308 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.399272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.399683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.399664    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"]
Jul 29 19:31:07.414102 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.414070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.416804 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.416777    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Jul 29 19:31:07.416953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.416888    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 29 19:31:07.416953 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.416934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Jul 29 19:31:07.417107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.416952    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-5zrdc\""
Jul 29 19:31:07.417107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.416892    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Jul 29 19:31:07.423429 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.423408    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"]
Jul 29 19:31:07.432049 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.432025    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hcp8p"]
Jul 29 19:31:07.435503 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.435483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.437524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.437173    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 29 19:31:07.437524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.437308    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 29 19:31:07.437524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.437379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qvrbw\""
Jul 29 19:31:07.437524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.437488    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 29 19:31:07.498520 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.498482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.529614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-textfile\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.529799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.529799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz24d\" (UniqueName: \"kubernetes.io/projected/6f902dac-3eaa-48f2-a226-7f392a185571-kube-api-access-nz24d\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.529799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.529799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-sys\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.530015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529818    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f902dac-3eaa-48f2-a226-7f392a185571-metrics-client-ca\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.530015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4kx7\" (UniqueName: \"kubernetes.io/projected/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-api-access-m4kx7\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.530015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.530015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-root\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.530015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-tls\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.530015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.529956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.530274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.530041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.530274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.530062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.530274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.530084    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.530274 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.530134    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-wtmp\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.612783 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.612693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.631007 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.630970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.631163 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz24d\" (UniqueName: \"kubernetes.io/projected/6f902dac-3eaa-48f2-a226-7f392a185571-kube-api-access-nz24d\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631252 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.631313 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-sys\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631368 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f902dac-3eaa-48f2-a226-7f392a185571-metrics-client-ca\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631422 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-sys\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631422 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4kx7\" (UniqueName: \"kubernetes.io/projected/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-api-access-m4kx7\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.631422 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631417    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631561 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631455    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-root\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631561 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631483    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-tls\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631561 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631521    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.631754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.631754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.631754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-wtmp\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.631754 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:07.631720    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Jul 29 19:31:07.631754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631737    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-volume-directive-shadow\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:07.631781    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-tls podName:f9deef4a-c592-4e2a-bd38-cc3ed62c5f67 nodeName:}" failed. No retries permitted until 2026-07-29 19:31:08.131761662 +0000 UTC m=+90.009615974 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-tls") pod "kube-state-metrics-56d9f944b4-c8dx7" (UID: "f9deef4a-c592-4e2a-bd38-cc3ed62c5f67") : secret "kube-state-metrics-tls" not found
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-wtmp\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631827    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/6f902dac-3eaa-48f2-a226-7f392a185571-root\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:07.631906    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:07.631938    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-tls podName:6f902dac-3eaa-48f2-a226-7f392a185571 nodeName:}" failed. No retries permitted until 2026-07-29 19:31:08.1319275 +0000 UTC m=+90.009781806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-tls") pod "node-exporter-hcp8p" (UID: "6f902dac-3eaa-48f2-a226-7f392a185571") : secret "node-exporter-tls" not found
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631976    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6f902dac-3eaa-48f2-a226-7f392a185571-metrics-client-ca\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.632015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-textfile\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.632156 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.631989    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-accelerators-collector-config\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.632576 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.632326    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-metrics-client-ca\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.632576 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.632355    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-textfile\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.632576 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.632474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.634477 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.634453    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.634477 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.634465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.643444 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.643411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz24d\" (UniqueName: \"kubernetes.io/projected/6f902dac-3eaa-48f2-a226-7f392a185571-kube-api-access-nz24d\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:07.643573 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.643528    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4kx7\" (UniqueName: \"kubernetes.io/projected/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-api-access-m4kx7\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:07.779909 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.779838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:07.959077 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:07.959037    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:08.138286 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.138246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-tls\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:08.138478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.138314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:08.141031 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.140980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/6f902dac-3eaa-48f2-a226-7f392a185571-node-exporter-tls\") pod \"node-exporter-hcp8p\" (UID: \"6f902dac-3eaa-48f2-a226-7f392a185571\") " pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:08.141160 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.141118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f9deef4a-c592-4e2a-bd38-cc3ed62c5f67-kube-state-metrics-tls\") pod \"kube-state-metrics-56d9f944b4-c8dx7\" (UID: \"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67\") " pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:08.326985 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.326906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"
Jul 29 19:31:08.346650 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.346623    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hcp8p"
Jul 29 19:31:08.564423 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.564392    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:31:08.570927 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.570898    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.573114 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573086    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n9f58\""
Jul 29 19:31:08.573252 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573161    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 19:31:08.573300 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 19:31:08.573332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573304    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 19:31:08.573383 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573371    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 19:31:08.573459 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573449    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 19:31:08.573555 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573535    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 19:31:08.573984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573811    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 19:31:08.573984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.573849    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 19:31:08.574458 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.574437    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 19:31:08.604251 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.604219    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:31:08.609883 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.609692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:08.643352 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643535 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643535 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643398    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643535 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643452    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643535 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643697 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643697 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-config-out\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643762 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643762 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643737    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-web-config\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643829 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643829 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-config-volume\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643894 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.643932 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.643908    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndlrn\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-kube-api-access-ndlrn\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744491 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744680 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744680 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-config-out\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744680 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744680 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-web-config\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744680 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-config-volume\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744727    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndlrn\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-kube-api-access-ndlrn\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.744933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.744889    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.745360 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.745334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.747955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.747925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.748731 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.748711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.749890 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.749856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-config-volume\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.750182 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.750136    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-config-out\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.750596 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.750575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.750737 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.750662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.751040 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.750987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.751159 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.751134    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-web-config\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.751695 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.751677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.752266 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.752244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.753155 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.753135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.758686 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.758664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndlrn\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-kube-api-access-ndlrn\") pod \"alertmanager-main-0\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:08.770746 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.770708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:08.873829 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.873743    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:08.883738 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:08.883706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:31:09.038452 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.038419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:09.040789 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:31:09.040755    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f902dac_3eaa_48f2_a226_7f392a185571.slice/crio-93198aef86a6570b1091202d3ee5bdac33beab1a597756e294742071d3747b30 WatchSource:0}: Error finding container 93198aef86a6570b1091202d3ee5bdac33beab1a597756e294742071d3747b30: Status 404 returned error can't find the container with id 93198aef86a6570b1091202d3ee5bdac33beab1a597756e294742071d3747b30
Jul 29 19:31:09.139826 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.139755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:09.147547 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.146693    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:31:09.221692 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.221415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:31:09.233536 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.233464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vrvjf" event={"ID":"0db0227f-3080-4a56-8950-bca5e9aeeea8","Type":"ContainerStarted","Data":"1a2793cb056de0dd5b553a6ce099fc2f238e57d481a04a60e2171348dfe0f051"}
Jul 29 19:31:09.235361 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.235311    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d576c7d69-q2jsg" event={"ID":"2bfba303-d00b-417a-bd89-bf0f1047ad35","Type":"ContainerStarted","Data":"59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7"}
Jul 29 19:31:09.237478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.237454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"87c097aa3564c88b001cdb8153177a27e37f226e64b230e14c2905a19889535f"}
Jul 29 19:31:09.238787 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.238743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcp8p" event={"ID":"6f902dac-3eaa-48f2-a226-7f392a185571","Type":"ContainerStarted","Data":"93198aef86a6570b1091202d3ee5bdac33beab1a597756e294742071d3747b30"}
Jul 29 19:31:09.246732 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.246656    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7"]
Jul 29 19:31:09.249539 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:31:09.249463    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9deef4a_c592_4e2a_bd38_cc3ed62c5f67.slice/crio-613e31fba9924bbd16f3ae848085a823a3223b73a9cbd0b1f9b941ffb116218e WatchSource:0}: Error finding container 613e31fba9924bbd16f3ae848085a823a3223b73a9cbd0b1f9b941ffb116218e: Status 404 returned error can't find the container with id 613e31fba9924bbd16f3ae848085a823a3223b73a9cbd0b1f9b941ffb116218e
Jul 29 19:31:09.272171 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.272108    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5d576c7d69-q2jsg" podStartSLOduration=1.566276003 podStartE2EDuration="8.272086259s" totalImagesPullingTime="6.705810256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:09.271363575 +0000 UTC m=+91.149217904" watchObservedRunningTime="2026-07-29 19:31:09.272086259 +0000 UTC m=+91.149940591"
Jul 29 19:31:09.445156 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.445116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:09.544917 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:09.544886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:10.246979 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:10.246890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7" event={"ID":"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67","Type":"ContainerStarted","Data":"613e31fba9924bbd16f3ae848085a823a3223b73a9cbd0b1f9b941ffb116218e"}
Jul 29 19:31:10.480493 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:10.480437    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:10.580383 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:10.580288    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:10.654222 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:10.654188    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:10.713522 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:10.713489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:11.034326 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:11.034292    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:12.134599 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.134564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:12.135127 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.134653    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:12.140169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.140141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:12.148374 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.148334    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:12.148541 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.148522    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"]
Jul 29 19:31:12.164690 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.164657    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"]
Jul 29 19:31:12.164870 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.164777    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.166897 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.166868    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-8qqx6\""
Jul 29 19:31:12.167148 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.167130    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 29 19:31:12.224068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.224024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:12.259466 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.259430    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:12.290029 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.289949    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzdvq\" (UniqueName: \"kubernetes.io/projected/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-kube-api-access-nzdvq\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.290222 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.290060    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.390938 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.390849    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.391134 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:12.390988    2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 29 19:31:12.391134 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:12.391076    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-monitoring-plugin-cert podName:24705fb5-d1aa-4b14-a8f3-fd6a83439acd nodeName:}" failed. No retries permitted until 2026-07-29 19:31:12.891052008 +0000 UTC m=+94.768906331 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-l9fkk" (UID: "24705fb5-d1aa-4b14-a8f3-fd6a83439acd") : secret "monitoring-plugin-cert" not found
Jul 29 19:31:12.391262 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.391149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzdvq\" (UniqueName: \"kubernetes.io/projected/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-kube-api-access-nzdvq\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.402654 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.402625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzdvq\" (UniqueName: \"kubernetes.io/projected/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-kube-api-access-nzdvq\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.444159 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.444124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:12.641656 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.641572    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8bcc8fd48-l9d99"]
Jul 29 19:31:12.648773 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.648739    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.658048 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.658016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bcc8fd48-l9d99"]
Jul 29 19:31:12.659464 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.659440    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 29 19:31:12.695130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-oauth-serving-cert\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.695309 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-console-config\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.695309 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spl5g\" (UniqueName: \"kubernetes.io/projected/27e9a06b-7e29-46dc-8897-662e9d73511d-kube-api-access-spl5g\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.695417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695301    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-serving-cert\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.695417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-oauth-config\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.695417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-trusted-ca-bundle\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.695417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.695401    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-service-ca\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796229 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796184    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-console-config\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spl5g\" (UniqueName: \"kubernetes.io/projected/27e9a06b-7e29-46dc-8897-662e9d73511d-kube-api-access-spl5g\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-serving-cert\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-oauth-config\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796569    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-trusted-ca-bundle\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796596    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-service-ca\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.796843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796737    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-oauth-serving-cert\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.797268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.796881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-console-config\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.797424 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.797402    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-trusted-ca-bundle\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.797424 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.797416    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-oauth-serving-cert\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.797541 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.797414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-service-ca\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.799510 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.799488    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-serving-cert\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.799788 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.799770    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-oauth-config\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.805409 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.805380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spl5g\" (UniqueName: \"kubernetes.io/projected/27e9a06b-7e29-46dc-8897-662e9d73511d-kube-api-access-spl5g\") pod \"console-8bcc8fd48-l9d99\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") " pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.900054 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.899958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.903011 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.902963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/24705fb5-d1aa-4b14-a8f3-fd6a83439acd-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-l9fkk\" (UID: \"24705fb5-d1aa-4b14-a8f3-fd6a83439acd\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:12.918202 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.918168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:12.966007 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.965965    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:12.987490 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:12.987460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:13.076886 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.076854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:13.263927 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.263807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7" event={"ID":"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67","Type":"ContainerStarted","Data":"ed29de90c9095191457067d47e79519f727fd89b211575c55c8e18cbfe9367ba"}
Jul 29 19:31:13.264428 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.264394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bcc8fd48-l9d99"]
Jul 29 19:31:13.266405 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.266325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vrvjf" event={"ID":"0db0227f-3080-4a56-8950-bca5e9aeeea8","Type":"ContainerStarted","Data":"2b2ed03350f580d29c524ecdcb1e1a700a7bfd8e7cf02f08c134c753cdcce8d4"}
Jul 29 19:31:13.268892 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.268780    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"}
Jul 29 19:31:13.271944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.271917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcp8p" event={"ID":"6f902dac-3eaa-48f2-a226-7f392a185571","Type":"ContainerStarted","Data":"c72ae653ac0477d9c4e8520c115ee041e31de2b5d64a18e5238c6c9ba86af1bd"}
Jul 29 19:31:13.273793 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:31:13.273716    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27e9a06b_7e29_46dc_8897_662e9d73511d.slice/crio-7b27302a6e71a363203b9c2f7f5eb2502170b2b705afa893130ca48b5a239081 WatchSource:0}: Error finding container 7b27302a6e71a363203b9c2f7f5eb2502170b2b705afa893130ca48b5a239081: Status 404 returned error can't find the container with id 7b27302a6e71a363203b9c2f7f5eb2502170b2b705afa893130ca48b5a239081
Jul 29 19:31:13.279694 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.279665    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"]
Jul 29 19:31:13.527020 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.526960    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:13.625009 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.624934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:13.774691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.774574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:13.845863 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:13.845803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:14.280796 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.280024    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcc8fd48-l9d99" event={"ID":"27e9a06b-7e29-46dc-8897-662e9d73511d","Type":"ContainerStarted","Data":"0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589"}
Jul 29 19:31:14.280796 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.280119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcc8fd48-l9d99" event={"ID":"27e9a06b-7e29-46dc-8897-662e9d73511d","Type":"ContainerStarted","Data":"7b27302a6e71a363203b9c2f7f5eb2502170b2b705afa893130ca48b5a239081"}
Jul 29 19:31:14.284945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.282960    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625" exitCode=0
Jul 29 19:31:14.284945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.283078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"}
Jul 29 19:31:14.286625 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.286547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk" event={"ID":"24705fb5-d1aa-4b14-a8f3-fd6a83439acd","Type":"ContainerStarted","Data":"8c00a0a53e5effd5984cfa9043f3d41c36ad7ad9a69ce21be6a819412685aaa1"}
Jul 29 19:31:14.288430 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.288398    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f902dac-3eaa-48f2-a226-7f392a185571" containerID="c72ae653ac0477d9c4e8520c115ee041e31de2b5d64a18e5238c6c9ba86af1bd" exitCode=0
Jul 29 19:31:14.288546 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.288500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcp8p" event={"ID":"6f902dac-3eaa-48f2-a226-7f392a185571","Type":"ContainerDied","Data":"c72ae653ac0477d9c4e8520c115ee041e31de2b5d64a18e5238c6c9ba86af1bd"}
Jul 29 19:31:14.291872 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.291838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7" event={"ID":"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67","Type":"ContainerStarted","Data":"f62cbf8f950f7af2e0f45a43b11d9cf4222812c77e9a23b8237ff106ef70d487"}
Jul 29 19:31:14.291969 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.291907    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7" event={"ID":"f9deef4a-c592-4e2a-bd38-cc3ed62c5f67","Type":"ContainerStarted","Data":"9e3602616e31874196c8c5d57bdb09875b1ed176672e48d6f60056c0b400fbd7"}
Jul 29 19:31:14.305848 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.305784    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8bcc8fd48-l9d99" podStartSLOduration=2.305758866 podStartE2EDuration="2.305758866s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:14.303699797 +0000 UTC m=+96.181554126" watchObservedRunningTime="2026-07-29 19:31:14.305758866 +0000 UTC m=+96.183613195"
Jul 29 19:31:14.358612 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:14.358111    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56d9f944b4-c8dx7" podStartSLOduration=3.524900085 podStartE2EDuration="7.358088122s" totalImagesPullingTime="3.833188037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:14.355926944 +0000 UTC m=+96.233781271" watchObservedRunningTime="2026-07-29 19:31:14.358088122 +0000 UTC m=+96.235942451"
Jul 29 19:31:15.113778 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:15.113740    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bmrsw"
Jul 29 19:31:15.301250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:15.300529    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcp8p" event={"ID":"6f902dac-3eaa-48f2-a226-7f392a185571","Type":"ContainerStarted","Data":"479bd96d224ee3f9b7217cbe0fe64d6278b84a85c564573eb8ac1f17803251aa"}
Jul 29 19:31:15.301250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:15.300589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hcp8p" event={"ID":"6f902dac-3eaa-48f2-a226-7f392a185571","Type":"ContainerStarted","Data":"0fc1405069af5efed68adfeea17dc3088cb0b4dfb31383f7775252035b5416d4"}
Jul 29 19:31:15.323698 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:15.323638    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hcp8p" podStartSLOduration=4.251909722 podStartE2EDuration="8.323620646s" totalImagesPullingTime="3.962512833s" totalInitContainerRuntime="109.198091ms" isStatefulPod=true podCreationTimestamp="2026-07-29 19:31:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:15.322322669 +0000 UTC m=+97.200177036" watchObservedRunningTime="2026-07-29 19:31:15.323620646 +0000 UTC m=+97.201474973"
Jul 29 19:31:15.435877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:15.435824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:15.510286 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:15.510247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:17.775524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:17.775485    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:17.871198 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:17.871113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:18.317607 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.317569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vrvjf" event={"ID":"0db0227f-3080-4a56-8950-bca5e9aeeea8","Type":"ContainerStarted","Data":"fbaf21c2dcdc72693efa91aecded76eaeee4c89efa2e3d1b913ca99568ab2e70"}
Jul 29 19:31:18.320743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.320709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"}
Jul 29 19:31:18.320884 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.320765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"}
Jul 29 19:31:18.320884 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.320800    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"}
Jul 29 19:31:18.320884 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.320827    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"}
Jul 29 19:31:18.320884 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.320865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"}
Jul 29 19:31:18.322253 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.322229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk" event={"ID":"24705fb5-d1aa-4b14-a8f3-fd6a83439acd","Type":"ContainerStarted","Data":"e14416675d32b43c3a70712d40ae7f7f74ab1d8b55157706c94e98ecc48e7ecb"}
Jul 29 19:31:18.322426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.322413    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:18.327545 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.327527    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk"
Jul 29 19:31:18.338966 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.338927    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vrvjf" podStartSLOduration=33.682388212 podStartE2EDuration="48.338914702s" totalImagesPullingTime="14.65652649s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 19:30:30 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:18.337706801 +0000 UTC m=+100.215561132" watchObservedRunningTime="2026-07-29 19:31:18.338914702 +0000 UTC m=+100.216769030"
Jul 29 19:31:18.355679 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:18.355630    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-l9fkk" podStartSLOduration=2.399554668 podStartE2EDuration="6.355609714s" totalImagesPullingTime="3.956055046s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:18.35483157 +0000 UTC m=+100.232685900" watchObservedRunningTime="2026-07-29 19:31:18.355609714 +0000 UTC m=+100.233464042"
Jul 29 19:31:19.004653 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:19.004618    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54fcb899b6-zz94f"]
Jul 29 19:31:19.124946 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:19.124910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:19.226937 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:19.226838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:19.316279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:19.316241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:19.329889 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:19.329851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerStarted","Data":"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"}
Jul 29 19:31:19.362442 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:19.362375    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.219062464 podStartE2EDuration="11.362356078s" totalImagesPullingTime="8.100310394s" totalInitContainerRuntime="42.98322ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:31:08 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:19.360766766 +0000 UTC m=+101.238621096" watchObservedRunningTime="2026-07-29 19:31:19.362356078 +0000 UTC m=+101.240210405"
Jul 29 19:31:22.966297 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:22.966242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:22.966684 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:22.966321    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:22.971244 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:22.971219    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:23.351248 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:23.351168    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:31:23.416315 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:23.416279    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d576c7d69-q2jsg"]
Jul 29 19:31:23.544235 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:23.544205    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:23.645299 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:23.645257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:25.189192 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:25.189144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:25.257310 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:25.257278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:34.391489 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:34.391460    2580 generic.go:356] "Generic (PLEG): container finished" podID="7894863b-2223-4519-b749-851b66facc7b" containerID="a78a2970927d149009933b299fceccf6f15abab1e90fadc90d9fe7d513e8db04" exitCode=0
Jul 29 19:31:34.391971 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:34.391535    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns" event={"ID":"7894863b-2223-4519-b749-851b66facc7b","Type":"ContainerDied","Data":"a78a2970927d149009933b299fceccf6f15abab1e90fadc90d9fe7d513e8db04"}
Jul 29 19:31:34.391971 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:34.391930    2580 scope.go:117] "RemoveContainer" containerID="a78a2970927d149009933b299fceccf6f15abab1e90fadc90d9fe7d513e8db04"
Jul 29 19:31:35.397569 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:35.397537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns" event={"ID":"7894863b-2223-4519-b749-851b66facc7b","Type":"ContainerStarted","Data":"1ca71a7a9cb926ebf133e9f1c34ad7057992bcb22cad89101b61abee1f1c19f8"}
Jul 29 19:31:41.187068 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:41.187030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:41.244108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:41.244073    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:44.024609 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.024573    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f" podUID="04ff50d9-45ce-4220-80e6-9a042447bde1" containerName="registry" containerID="cri-o://58694955d1c2c295ea421ba6f9226519927f3ae350864b17d0974b91b5a05eed" gracePeriod=30
Jul 29 19:31:44.434876 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.434839    2580 generic.go:356] "Generic (PLEG): container finished" podID="04ff50d9-45ce-4220-80e6-9a042447bde1" containerID="58694955d1c2c295ea421ba6f9226519927f3ae350864b17d0974b91b5a05eed" exitCode=0
Jul 29 19:31:44.435066 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.434914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f" event={"ID":"04ff50d9-45ce-4220-80e6-9a042447bde1","Type":"ContainerDied","Data":"58694955d1c2c295ea421ba6f9226519927f3ae350864b17d0974b91b5a05eed"}
Jul 29 19:31:44.856367 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.856335    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:31:44.897603 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897572    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-certificates\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897630    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-installation-pull-secrets\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897708    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-bound-sa-token\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897735    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-image-registry-private-configuration\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897750    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-trusted-ca\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04ff50d9-45ce-4220-80e6-9a042447bde1-ca-trust-extracted\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psph2\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-kube-api-access-psph2\") pod \"04ff50d9-45ce-4220-80e6-9a042447bde1\" (UID: \"04ff50d9-45ce-4220-80e6-9a042447bde1\") "
Jul 29 19:31:44.897984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.897936    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:31:44.898514 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.898311    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-certificates\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.898514 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.898467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:31:44.900577 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.900532    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:31:44.900657 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.900573    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:31:44.900824 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.900795    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:31:44.900938 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.900860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-kube-api-access-psph2" (OuterVolumeSpecName: "kube-api-access-psph2") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "kube-api-access-psph2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:31:44.901039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.901014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:31:44.906559 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.906538    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ff50d9-45ce-4220-80e6-9a042447bde1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "04ff50d9-45ce-4220-80e6-9a042447bde1" (UID: "04ff50d9-45ce-4220-80e6-9a042447bde1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999676    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-image-registry-private-configuration\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999704    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/04ff50d9-45ce-4220-80e6-9a042447bde1-trusted-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999715    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/04ff50d9-45ce-4220-80e6-9a042447bde1-ca-trust-extracted\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999724    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psph2\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-kube-api-access-psph2\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999735    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/04ff50d9-45ce-4220-80e6-9a042447bde1-installation-pull-secrets\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999743    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-registry-tls\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:44.999757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:44.999750    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/04ff50d9-45ce-4220-80e6-9a042447bde1-bound-sa-token\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:45.428712 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.428676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:45.440154 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.440124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f" event={"ID":"04ff50d9-45ce-4220-80e6-9a042447bde1","Type":"ContainerDied","Data":"b87c46d18e21c4f8489268e60175e6b8758cca3a702312c31c6e254e290c74f0"}
Jul 29 19:31:45.440154 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.440151    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-54fcb899b6-zz94f"
Jul 29 19:31:45.440314 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.440185    2580 scope.go:117] "RemoveContainer" containerID="58694955d1c2c295ea421ba6f9226519927f3ae350864b17d0974b91b5a05eed"
Jul 29 19:31:45.464805 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.464757    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-54fcb899b6-zz94f"]
Jul 29 19:31:45.473642 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.473612    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-54fcb899b6-zz94f"]
Jul 29 19:31:45.531018 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:45.530960    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:46.714403 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:46.714367    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ff50d9-45ce-4220-80e6-9a042447bde1" path="/var/lib/kubelet/pods/04ff50d9-45ce-4220-80e6-9a042447bde1/volumes"
Jul 29 19:31:47.522180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:47.522141    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:31:47.524638 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:47.524612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/25377374-58d1-4eab-be9b-6a35696cc589-metrics-certs\") pod \"network-metrics-daemon-rx6zl\" (UID: \"25377374-58d1-4eab-be9b-6a35696cc589\") " pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:31:47.532372 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:47.532354    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-xr6nn\""
Jul 29 19:31:47.541588 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:47.541569    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rx6zl"
Jul 29 19:31:47.792483 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:47.792459    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rx6zl"]
Jul 29 19:31:47.794358 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:31:47.794327    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25377374_58d1_4eab_be9b_6a35696cc589.slice/crio-5e8f9b2b00984f644b41f2552836b052b168afc73bd65e8f65ad8b9592408199 WatchSource:0}: Error finding container 5e8f9b2b00984f644b41f2552836b052b168afc73bd65e8f65ad8b9592408199: Status 404 returned error can't find the container with id 5e8f9b2b00984f644b41f2552836b052b168afc73bd65e8f65ad8b9592408199
Jul 29 19:31:48.436917 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.436878    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5d576c7d69-q2jsg" podUID="2bfba303-d00b-417a-bd89-bf0f1047ad35" containerName="console" containerID="cri-o://59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7" gracePeriod=15
Jul 29 19:31:48.460055 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.460016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx6zl" event={"ID":"25377374-58d1-4eab-be9b-6a35696cc589","Type":"ContainerStarted","Data":"5e8f9b2b00984f644b41f2552836b052b168afc73bd65e8f65ad8b9592408199"}
Jul 29 19:31:48.722461 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.722421    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:48.834538 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.834506    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-config\") pod \"2bfba303-d00b-417a-bd89-bf0f1047ad35\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") "
Jul 29 19:31:48.834951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.834571    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-oauth-serving-cert\") pod \"2bfba303-d00b-417a-bd89-bf0f1047ad35\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") "
Jul 29 19:31:48.834951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.834597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-service-ca\") pod \"2bfba303-d00b-417a-bd89-bf0f1047ad35\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") "
Jul 29 19:31:48.834951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.834628    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jjhcs\" (UniqueName: \"kubernetes.io/projected/2bfba303-d00b-417a-bd89-bf0f1047ad35-kube-api-access-jjhcs\") pod \"2bfba303-d00b-417a-bd89-bf0f1047ad35\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") "
Jul 29 19:31:48.834951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.834673    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-serving-cert\") pod \"2bfba303-d00b-417a-bd89-bf0f1047ad35\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") "
Jul 29 19:31:48.834951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.834688    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-oauth-config\") pod \"2bfba303-d00b-417a-bd89-bf0f1047ad35\" (UID: \"2bfba303-d00b-417a-bd89-bf0f1047ad35\") "
Jul 29 19:31:48.835226 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.835031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-config" (OuterVolumeSpecName: "console-config") pod "2bfba303-d00b-417a-bd89-bf0f1047ad35" (UID: "2bfba303-d00b-417a-bd89-bf0f1047ad35"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:31:48.835226 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.835036    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2bfba303-d00b-417a-bd89-bf0f1047ad35" (UID: "2bfba303-d00b-417a-bd89-bf0f1047ad35"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:31:48.835226 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.835125    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-service-ca" (OuterVolumeSpecName: "service-ca") pod "2bfba303-d00b-417a-bd89-bf0f1047ad35" (UID: "2bfba303-d00b-417a-bd89-bf0f1047ad35"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:31:48.837140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.837103    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bfba303-d00b-417a-bd89-bf0f1047ad35-kube-api-access-jjhcs" (OuterVolumeSpecName: "kube-api-access-jjhcs") pod "2bfba303-d00b-417a-bd89-bf0f1047ad35" (UID: "2bfba303-d00b-417a-bd89-bf0f1047ad35"). InnerVolumeSpecName "kube-api-access-jjhcs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:31:48.837259 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.837141    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2bfba303-d00b-417a-bd89-bf0f1047ad35" (UID: "2bfba303-d00b-417a-bd89-bf0f1047ad35"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:31:48.837259 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.837188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2bfba303-d00b-417a-bd89-bf0f1047ad35" (UID: "2bfba303-d00b-417a-bd89-bf0f1047ad35"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:31:48.935180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.935153    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjhcs\" (UniqueName: \"kubernetes.io/projected/2bfba303-d00b-417a-bd89-bf0f1047ad35-kube-api-access-jjhcs\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:48.935180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.935179    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:48.935362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.935188    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-oauth-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:48.935362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.935198    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-console-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:48.935362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.935206    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-oauth-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:48.935362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:48.935214    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2bfba303-d00b-417a-bd89-bf0f1047ad35-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:31:49.465460 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.465429    2580 generic.go:356] "Generic (PLEG): container finished" podID="8b64ee3f-af74-4309-a2c2-e3071335f57d" containerID="0e2bf63cfbe9193511511925f437fafa58e29558bedcaa9141a857496b6ebba1" exitCode=0
Jul 29 19:31:49.465658 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.465512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh" event={"ID":"8b64ee3f-af74-4309-a2c2-e3071335f57d","Type":"ContainerDied","Data":"0e2bf63cfbe9193511511925f437fafa58e29558bedcaa9141a857496b6ebba1"}
Jul 29 19:31:49.465904 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.465888    2580 scope.go:117] "RemoveContainer" containerID="0e2bf63cfbe9193511511925f437fafa58e29558bedcaa9141a857496b6ebba1"
Jul 29 19:31:49.466817 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.466797    2580 generic.go:356] "Generic (PLEG): container finished" podID="2bfba303-d00b-417a-bd89-bf0f1047ad35" containerID="59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7" exitCode=0
Jul 29 19:31:49.466903 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.466867    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d576c7d69-q2jsg" event={"ID":"2bfba303-d00b-417a-bd89-bf0f1047ad35","Type":"ContainerDied","Data":"59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7"}
Jul 29 19:31:49.466963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.466904    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5d576c7d69-q2jsg" event={"ID":"2bfba303-d00b-417a-bd89-bf0f1047ad35","Type":"ContainerDied","Data":"a263449884935bbd8a77447bbc93c9a5294e6c53059ae0fb3b77964cfca09e91"}
Jul 29 19:31:49.466963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.466910    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5d576c7d69-q2jsg"
Jul 29 19:31:49.466963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.466928    2580 scope.go:117] "RemoveContainer" containerID="59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7"
Jul 29 19:31:49.468888 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.468862    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx6zl" event={"ID":"25377374-58d1-4eab-be9b-6a35696cc589","Type":"ContainerStarted","Data":"f9f420da66abd9b34aea11396e131bdeec6277c221173be38c1939c34925efc1"}
Jul 29 19:31:49.468961 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.468923    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rx6zl" event={"ID":"25377374-58d1-4eab-be9b-6a35696cc589","Type":"ContainerStarted","Data":"671d362e0f5436ba33a82df04966dcc55bcf5de8e28b4fab90e0282e62147a83"}
Jul 29 19:31:49.475582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.475568    2580 scope.go:117] "RemoveContainer" containerID="59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7"
Jul 29 19:31:49.475803 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:31:49.475777    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7\": container with ID starting with 59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7 not found: ID does not exist" containerID="59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7"
Jul 29 19:31:49.475845 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.475811    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7"} err="failed to get container status \"59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7\": rpc error: code = NotFound desc = could not find container \"59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7\": container with ID starting with 59fd1609ddca1d505768fd1864a53f0abf8b55a73fc9d4ca15b2f9efd4d2afa7 not found: ID does not exist"
Jul 29 19:31:49.520230 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.520173    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rx6zl" podStartSLOduration=130.731782015 podStartE2EDuration="2m11.520150555s" totalImagesPullingTime="788.36854ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:29:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:31:49.518851779 +0000 UTC m=+131.396706109" watchObservedRunningTime="2026-07-29 19:31:49.520150555 +0000 UTC m=+131.398004885"
Jul 29 19:31:49.552271 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.552231    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5d576c7d69-q2jsg"]
Jul 29 19:31:49.559545 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.559519    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5d576c7d69-q2jsg"]
Jul 29 19:31:49.692456 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.692419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:49.750019 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:49.749914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:31:50.475066 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:50.475031    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-c4pmh" event={"ID":"8b64ee3f-af74-4309-a2c2-e3071335f57d","Type":"ContainerStarted","Data":"94b0b721c5d58d8dc6d8961f7debd0b1bdcffcb41bf025177439fd0bd419618b"}
Jul 29 19:31:50.720113 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:50.720079    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2bfba303-d00b-417a-bd89-bf0f1047ad35" path="/var/lib/kubelet/pods/2bfba303-d00b-417a-bd89-bf0f1047ad35/volumes"
Jul 29 19:31:59.513376 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:59.513344    2580 generic.go:356] "Generic (PLEG): container finished" podID="588349d1-be21-4927-b936-0ba5f6e725b3" containerID="79617b526d3fa78387879a31e65b5b477991d9627f3df7b488f62f11a6340caa" exitCode=0
Jul 29 19:31:59.513676 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:59.513420    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-dlrtg" event={"ID":"588349d1-be21-4927-b936-0ba5f6e725b3","Type":"ContainerDied","Data":"79617b526d3fa78387879a31e65b5b477991d9627f3df7b488f62f11a6340caa"}
Jul 29 19:31:59.513807 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:31:59.513791    2580 scope.go:117] "RemoveContainer" containerID="79617b526d3fa78387879a31e65b5b477991d9627f3df7b488f62f11a6340caa"
Jul 29 19:32:00.519523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:00.519486    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-dlrtg" event={"ID":"588349d1-be21-4927-b936-0ba5f6e725b3","Type":"ContainerStarted","Data":"8ab251ae175ed6b2b77746f929cd05666fe7ffec04ba7822a85962ffec927a03"}
Jul 29 19:32:14.620582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:14.620543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:26.885764 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:26.885727    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:27.010796 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.010764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:27.923406 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923373    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:32:27.923808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923680    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="alertmanager" containerID="cri-o://5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546" gracePeriod=120
Jul 29 19:32:27.923808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923740    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-metric" containerID="cri-o://79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa" gracePeriod=120
Jul 29 19:32:27.923972 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923924    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="config-reloader" containerID="cri-o://6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7" gracePeriod=120
Jul 29 19:32:27.924136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923761    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-web" containerID="cri-o://59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a" gracePeriod=120
Jul 29 19:32:27.924136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923766    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy" containerID="cri-o://9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c" gracePeriod=120
Jul 29 19:32:27.924136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:27.923827    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="prom-label-proxy" containerID="cri-o://dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e" gracePeriod=120
Jul 29 19:32:28.635606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635568    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e" exitCode=0
Jul 29 19:32:28.635606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635596    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c" exitCode=0
Jul 29 19:32:28.635606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635607    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7" exitCode=0
Jul 29 19:32:28.635606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635615    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546" exitCode=0
Jul 29 19:32:28.635928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635652    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"}
Jul 29 19:32:28.635928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"}
Jul 29 19:32:28.635928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"}
Jul 29 19:32:28.635928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:28.635804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"}
Jul 29 19:32:29.170013 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.169974    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.278956 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.278869    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-metric\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.278956 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.278908    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-web-config\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.278956 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.278932    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-main-tls\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.278967    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-trusted-ca-bundle\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279026    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-web\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ndlrn\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-kube-api-access-ndlrn\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279101    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-main-db\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279124    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-config-volume\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279155    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-tls-assets\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279193    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-config-out\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279236    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-cluster-tls-config\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279270    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-metrics-client-ca\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279903 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279309    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy\") pod \"ea842368-b65f-461c-a1d8-b88e17aa970b\" (UID: \"ea842368-b65f-461c-a1d8-b88e17aa970b\") "
Jul 29 19:32:29.279903 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279433    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:32:29.279903 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.279550    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.280325 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.280089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:32:29.282221 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.282195    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.282639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.282445    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-kube-api-access-ndlrn" (OuterVolumeSpecName: "kube-api-access-ndlrn") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "kube-api-access-ndlrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:32:29.282732 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.282693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.282732 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.282716    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.282956 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.282898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:32:29.283072 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.282983    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:32:29.283278 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.283251    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.285038 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.285008    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-config-out" (OuterVolumeSpecName: "config-out") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:32:29.285155 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.285140    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-config-volume" (OuterVolumeSpecName: "config-volume") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.287340 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.287319    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.293900 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.293875    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-web-config" (OuterVolumeSpecName: "web-config") pod "ea842368-b65f-461c-a1d8-b88e17aa970b" (UID: "ea842368-b65f-461c-a1d8-b88e17aa970b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:32:29.380064 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380027    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380064 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380062    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-web-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380073    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-main-tls\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380084    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380093    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ndlrn\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-kube-api-access-ndlrn\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380102    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-alertmanager-main-db\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380111    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-config-volume\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380119    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ea842368-b65f-461c-a1d8-b88e17aa970b-tls-assets\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380129    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ea842368-b65f-461c-a1d8-b88e17aa970b-config-out\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380135    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-cluster-tls-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380143    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ea842368-b65f-461c-a1d8-b88e17aa970b-metrics-client-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.380246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.380151    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ea842368-b65f-461c-a1d8-b88e17aa970b-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:32:29.642398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642368    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa" exitCode=0
Jul 29 19:32:29.642398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642392    2580 generic.go:356] "Generic (PLEG): container finished" podID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerID="59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a" exitCode=0
Jul 29 19:32:29.642639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642445    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.642639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"}
Jul 29 19:32:29.642639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"}
Jul 29 19:32:29.642639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ea842368-b65f-461c-a1d8-b88e17aa970b","Type":"ContainerDied","Data":"87c097aa3564c88b001cdb8153177a27e37f226e64b230e14c2905a19889535f"}
Jul 29 19:32:29.642639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.642587    2580 scope.go:117] "RemoveContainer" containerID="dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"
Jul 29 19:32:29.650643 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.650624    2580 scope.go:117] "RemoveContainer" containerID="79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"
Jul 29 19:32:29.657921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.657905    2580 scope.go:117] "RemoveContainer" containerID="9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"
Jul 29 19:32:29.665043 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.665019    2580 scope.go:117] "RemoveContainer" containerID="59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"
Jul 29 19:32:29.673877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.673851    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:32:29.675266 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.675245    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:32:29.675715 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.675694    2580 scope.go:117] "RemoveContainer" containerID="6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"
Jul 29 19:32:29.683336 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.683313    2580 scope.go:117] "RemoveContainer" containerID="5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"
Jul 29 19:32:29.690476 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.690463    2580 scope.go:117] "RemoveContainer" containerID="c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"
Jul 29 19:32:29.697549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.697534    2580 scope.go:117] "RemoveContainer" containerID="dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"
Jul 29 19:32:29.697793 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.697775    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e\": container with ID starting with dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e not found: ID does not exist" containerID="dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"
Jul 29 19:32:29.697846 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.697801    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"} err="failed to get container status \"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e\": rpc error: code = NotFound desc = could not find container \"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e\": container with ID starting with dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e not found: ID does not exist"
Jul 29 19:32:29.697846 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.697820    2580 scope.go:117] "RemoveContainer" containerID="79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"
Jul 29 19:32:29.698031 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.698015    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa\": container with ID starting with 79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa not found: ID does not exist" containerID="79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"
Jul 29 19:32:29.698075 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698035    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"} err="failed to get container status \"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa\": rpc error: code = NotFound desc = could not find container \"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa\": container with ID starting with 79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa not found: ID does not exist"
Jul 29 19:32:29.698075 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698049    2580 scope.go:117] "RemoveContainer" containerID="9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"
Jul 29 19:32:29.698263 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.698248    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c\": container with ID starting with 9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c not found: ID does not exist" containerID="9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"
Jul 29 19:32:29.698310 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698266    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"} err="failed to get container status \"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c\": rpc error: code = NotFound desc = could not find container \"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c\": container with ID starting with 9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c not found: ID does not exist"
Jul 29 19:32:29.698310 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698277    2580 scope.go:117] "RemoveContainer" containerID="59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"
Jul 29 19:32:29.698482 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.698467    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a\": container with ID starting with 59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a not found: ID does not exist" containerID="59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"
Jul 29 19:32:29.698519 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698485    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"} err="failed to get container status \"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a\": rpc error: code = NotFound desc = could not find container \"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a\": container with ID starting with 59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a not found: ID does not exist"
Jul 29 19:32:29.698519 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698496    2580 scope.go:117] "RemoveContainer" containerID="6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"
Jul 29 19:32:29.698684 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.698671    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7\": container with ID starting with 6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7 not found: ID does not exist" containerID="6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"
Jul 29 19:32:29.698755 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698686    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"} err="failed to get container status \"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7\": rpc error: code = NotFound desc = could not find container \"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7\": container with ID starting with 6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7 not found: ID does not exist"
Jul 29 19:32:29.698755 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698697    2580 scope.go:117] "RemoveContainer" containerID="5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"
Jul 29 19:32:29.698892 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.698873    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546\": container with ID starting with 5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546 not found: ID does not exist" containerID="5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"
Jul 29 19:32:29.698930 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698896    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"} err="failed to get container status \"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546\": rpc error: code = NotFound desc = could not find container \"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546\": container with ID starting with 5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546 not found: ID does not exist"
Jul 29 19:32:29.698930 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.698907    2580 scope.go:117] "RemoveContainer" containerID="c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"
Jul 29 19:32:29.699123 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:32:29.699102    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625\": container with ID starting with c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625 not found: ID does not exist" containerID="c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"
Jul 29 19:32:29.699167 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699135    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"} err="failed to get container status \"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625\": rpc error: code = NotFound desc = could not find container \"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625\": container with ID starting with c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625 not found: ID does not exist"
Jul 29 19:32:29.699167 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699152    2580 scope.go:117] "RemoveContainer" containerID="dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"
Jul 29 19:32:29.699360 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699344    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e"} err="failed to get container status \"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e\": rpc error: code = NotFound desc = could not find container \"dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e\": container with ID starting with dfd9b0c1d402eda2d9c86c050fcbd573a1568bdb060d7f0f3c33f952307b2f7e not found: ID does not exist"
Jul 29 19:32:29.699397 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699361    2580 scope.go:117] "RemoveContainer" containerID="79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"
Jul 29 19:32:29.699582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699567    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa"} err="failed to get container status \"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa\": rpc error: code = NotFound desc = could not find container \"79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa\": container with ID starting with 79beda474a618a8caad6f2ee74044204324338e1cba7dcfc4743f9d4ffb49bfa not found: ID does not exist"
Jul 29 19:32:29.699624 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699582    2580 scope.go:117] "RemoveContainer" containerID="9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"
Jul 29 19:32:29.699753 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699733    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c"} err="failed to get container status \"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c\": rpc error: code = NotFound desc = could not find container \"9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c\": container with ID starting with 9d52a9122c906706bec20782d0bb69b497fae7dfb72fb7d8819526271d9f484c not found: ID does not exist"
Jul 29 19:32:29.699801 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699755    2580 scope.go:117] "RemoveContainer" containerID="59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"
Jul 29 19:32:29.699944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699929    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a"} err="failed to get container status \"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a\": rpc error: code = NotFound desc = could not find container \"59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a\": container with ID starting with 59048f1a007e10ab1bdbb7c0abc31d9495bd086c955d021b9f6fddf53485050a not found: ID does not exist"
Jul 29 19:32:29.699944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.699943    2580 scope.go:117] "RemoveContainer" containerID="6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"
Jul 29 19:32:29.700107 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.700090    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7"} err="failed to get container status \"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7\": rpc error: code = NotFound desc = could not find container \"6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7\": container with ID starting with 6f2253cb0b699d9470e55814c97ea5dfdd570e40538b53098c2102ac791179f7 not found: ID does not exist"
Jul 29 19:32:29.700146 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.700106    2580 scope.go:117] "RemoveContainer" containerID="5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"
Jul 29 19:32:29.700305 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.700286    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546"} err="failed to get container status \"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546\": rpc error: code = NotFound desc = could not find container \"5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546\": container with ID starting with 5e8f1d516bdc23c49f934dbbca7eb04bcb4a0a541778aea4b8db39a6dec32546 not found: ID does not exist"
Jul 29 19:32:29.700345 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.700307    2580 scope.go:117] "RemoveContainer" containerID="c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"
Jul 29 19:32:29.700518 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.700502    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625"} err="failed to get container status \"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625\": rpc error: code = NotFound desc = could not find container \"c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625\": container with ID starting with c02ac0d34145a5419868c0bbf8a030713b9c44e8ee7e12cba0daeb064e7c9625 not found: ID does not exist"
Jul 29 19:32:29.703574 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703552    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:32:29.703876 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703863    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-web"
Jul 29 19:32:29.703921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703877    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-web"
Jul 29 19:32:29.703921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703886    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="prom-label-proxy"
Jul 29 19:32:29.703921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703892    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="prom-label-proxy"
Jul 29 19:32:29.703921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703903    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="config-reloader"
Jul 29 19:32:29.703921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703911    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="config-reloader"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703929    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="alertmanager"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703935    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="alertmanager"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703941    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="init-config-reloader"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703946    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="init-config-reloader"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703952    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703957    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703962    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ff50d9-45ce-4220-80e6-9a042447bde1" containerName="registry"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703967    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ff50d9-45ce-4220-80e6-9a042447bde1" containerName="registry"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703972    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bfba303-d00b-417a-bd89-bf0f1047ad35" containerName="console"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703977    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bfba303-d00b-417a-bd89-bf0f1047ad35" containerName="console"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703982    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-metric"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.703988    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-metric"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704060    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-web"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704068    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy-metric"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704073    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="prom-label-proxy"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704082    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ff50d9-45ce-4220-80e6-9a042447bde1" containerName="registry"
Jul 29 19:32:29.704089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704091    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="config-reloader"
Jul 29 19:32:29.704544 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704098    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="kube-rbac-proxy"
Jul 29 19:32:29.704544 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704103    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" containerName="alertmanager"
Jul 29 19:32:29.704544 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.704109    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bfba303-d00b-417a-bd89-bf0f1047ad35" containerName="console"
Jul 29 19:32:29.707205 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.707188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.709277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709248    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Jul 29 19:32:29.709277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Jul 29 19:32:29.709421 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709251    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-n9f58\""
Jul 29 19:32:29.709421 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Jul 29 19:32:29.709589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709571    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Jul 29 19:32:29.709589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709584    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Jul 29 19:32:29.709589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709571    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Jul 29 19:32:29.709916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709899    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Jul 29 19:32:29.710013 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.709980    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Jul 29 19:32:29.714318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.714300    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Jul 29 19:32:29.724384 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.724364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:32:29.783767 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783731    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783767 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-web-config\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783787    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwtk\" (UniqueName: \"kubernetes.io/projected/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-kube-api-access-qqwtk\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.783974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783965    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-config-out\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.784214 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.783981    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.784214 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.784029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.784214 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.784050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.784214 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.784081    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885227 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885189    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885227 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885441 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885441 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-config-out\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885441 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885441 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885389    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885441 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885512    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-web-config\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwtk\" (UniqueName: \"kubernetes.io/projected/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-kube-api-access-qqwtk\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.885984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.885683    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.886532 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.886504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.886629 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.886542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.888488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.888365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-config-out\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.888668 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.888624    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-tls-assets\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.889277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.888757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.889277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.888817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-config-volume\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.889277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.888949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.889277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.889030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.889277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.889221    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.889502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.889323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-web-config\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.890741 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.890720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:29.896809 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:29.896768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwtk\" (UniqueName: \"kubernetes.io/projected/507ab561-0f67-4cb0-9c6a-6d06cc9c22c2-kube-api-access-qqwtk\") pod \"alertmanager-main-0\" (UID: \"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2\") " pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:30.016717 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:30.016680    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Jul 29 19:32:30.154853 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:30.154827    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Jul 29 19:32:30.157368 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:32:30.157336    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod507ab561_0f67_4cb0_9c6a_6d06cc9c22c2.slice/crio-970a83d7889030d4118413cd0b2dc112c080c787b09fb6bc4c21a35b5fac5cf0 WatchSource:0}: Error finding container 970a83d7889030d4118413cd0b2dc112c080c787b09fb6bc4c21a35b5fac5cf0: Status 404 returned error can't find the container with id 970a83d7889030d4118413cd0b2dc112c080c787b09fb6bc4c21a35b5fac5cf0
Jul 29 19:32:30.648344 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:30.648307    2580 generic.go:356] "Generic (PLEG): container finished" podID="507ab561-0f67-4cb0-9c6a-6d06cc9c22c2" containerID="ddb0a1493c2a4283c913da15c434ff35ea76bdeb362ba1deb3d6e5982b08dafc" exitCode=0
Jul 29 19:32:30.648801 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:30.648392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerDied","Data":"ddb0a1493c2a4283c913da15c434ff35ea76bdeb362ba1deb3d6e5982b08dafc"}
Jul 29 19:32:30.648801 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:30.648464    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"970a83d7889030d4118413cd0b2dc112c080c787b09fb6bc4c21a35b5fac5cf0"}
Jul 29 19:32:30.715306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:30.715277    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea842368-b65f-461c-a1d8-b88e17aa970b" path="/var/lib/kubelet/pods/ea842368-b65f-461c-a1d8-b88e17aa970b/volumes"
Jul 29 19:32:31.657393 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.657356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"2a2f9201f5b51508178a0e64a48038f70c01b82ca4ecf7c596c7cc6221f58e01"}
Jul 29 19:32:31.657766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.657449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"ad86eddcd50109b8aa1f557838d236c62174bed3e4813739a35f10441fce8e59"}
Jul 29 19:32:31.657766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.657475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"11b03ffd0544d6ec5bb976907d064e084e00b1d51a9011e1c644e1a778520f55"}
Jul 29 19:32:31.657766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.657504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"e87386b460feb413bbc405569b35847db96587eda1a511d4a93b8df36c9687e2"}
Jul 29 19:32:31.657766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.657529    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"3af1c93528e5951b1220a3499dab6dea795f429dbc765c71395abc54ad82e346"}
Jul 29 19:32:31.657766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.657552    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"507ab561-0f67-4cb0-9c6a-6d06cc9c22c2","Type":"ContainerStarted","Data":"dcba5f57954992754b2e15cd3a075d20fe9caebf2283b227a1c8114ce5dbe39c"}
Jul 29 19:32:31.686488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.686437    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.670693623 podStartE2EDuration="2.686421637s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.728014ms" isStatefulPod=false podCreationTimestamp="2026-07-29 19:32:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:32:31.68454082 +0000 UTC m=+173.562395165" watchObservedRunningTime="2026-07-29 19:32:31.686421637 +0000 UTC m=+173.564275965"
Jul 29 19:32:31.995443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.995368    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"]
Jul 29 19:32:31.997863 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.997845    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:31.999770 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.999744    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-c7gk8\""
Jul 29 19:32:31.999887 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.999768    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Jul 29 19:32:31.999887 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:31.999768    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Jul 29 19:32:32.000188 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.000157    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Jul 29 19:32:32.000188 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.000174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Jul 29 19:32:32.000381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.000160    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Jul 29 19:32:32.005047 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.005024    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Jul 29 19:32:32.012984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.012964    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"]
Jul 29 19:32:32.104183 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104140    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-metrics-client-ca\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-telemeter-client-tls\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104326    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-secret-telemeter-client\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-federate-client-tls\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-serving-certs-ca-bundle\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.104582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.104426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lsff\" (UniqueName: \"kubernetes.io/projected/63c2f63f-85d3-40a9-ae85-31190b462604-kube-api-access-6lsff\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205226 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lsff\" (UniqueName: \"kubernetes.io/projected/63c2f63f-85d3-40a9-ae85-31190b462604-kube-api-access-6lsff\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205250    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205277    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-metrics-client-ca\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-telemeter-client-tls\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205343    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-secret-telemeter-client\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-federate-client-tls\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.205418 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.205385    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-serving-certs-ca-bundle\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.206188 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.206159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-serving-certs-ca-bundle\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.206476 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.206449    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-metrics-client-ca\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.206582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.206566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/63c2f63f-85d3-40a9-ae85-31190b462604-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.208037 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.207987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-telemeter-client-tls\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.208128 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.208076    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-secret-telemeter-client\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.208166 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.208150    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.208362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.208344    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/63c2f63f-85d3-40a9-ae85-31190b462604-federate-client-tls\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.213954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.213936    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lsff\" (UniqueName: \"kubernetes.io/projected/63c2f63f-85d3-40a9-ae85-31190b462604-kube-api-access-6lsff\") pod \"telemeter-client-5f9dbf67db-bkp9l\" (UID: \"63c2f63f-85d3-40a9-ae85-31190b462604\") " pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.308438 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.308360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"
Jul 29 19:32:32.444809 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.444779    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l"]
Jul 29 19:32:32.447042 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:32:32.447012    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c2f63f_85d3_40a9_ae85_31190b462604.slice/crio-8b42de22a5804f9004398efd6d998ce2bc6aeb82e71ca6ba5741c1130608dfd3 WatchSource:0}: Error finding container 8b42de22a5804f9004398efd6d998ce2bc6aeb82e71ca6ba5741c1130608dfd3: Status 404 returned error can't find the container with id 8b42de22a5804f9004398efd6d998ce2bc6aeb82e71ca6ba5741c1130608dfd3
Jul 29 19:32:32.662362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:32.662328    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l" event={"ID":"63c2f63f-85d3-40a9-ae85-31190b462604","Type":"ContainerStarted","Data":"8b42de22a5804f9004398efd6d998ce2bc6aeb82e71ca6ba5741c1130608dfd3"}
Jul 29 19:32:34.679952 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:34.679908    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l" event={"ID":"63c2f63f-85d3-40a9-ae85-31190b462604","Type":"ContainerStarted","Data":"06660b35b75ac1109e7447bffd98c942653e7956b30d8990557196b7b1c242fb"}
Jul 29 19:32:34.680457 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:34.680028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l" event={"ID":"63c2f63f-85d3-40a9-ae85-31190b462604","Type":"ContainerStarted","Data":"f7f03289fb9a5f1bfdc0c34cf09ee6f79d13eb9dd664fd0591a25324c729769b"}
Jul 29 19:32:34.680457 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:34.680060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l" event={"ID":"63c2f63f-85d3-40a9-ae85-31190b462604","Type":"ContainerStarted","Data":"ec96cbf8f14f0fe33b2b6c2a7cf772e474e770019b7ec3db7f02fd93530cf8e7"}
Jul 29 19:32:34.741877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:34.741824    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5f9dbf67db-bkp9l" podStartSLOduration=1.8177101119999999 podStartE2EDuration="3.741809663s" totalImagesPullingTime="1.924099551s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:32:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:32:34.74055043 +0000 UTC m=+176.618404770" watchObservedRunningTime="2026-07-29 19:32:34.741809663 +0000 UTC m=+176.619663990"
Jul 29 19:32:35.512180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.512148    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58fb78d5bd-tdxkb"]
Jul 29 19:32:35.515305 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.515281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.571063 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.571031    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58fb78d5bd-tdxkb"]
Jul 29 19:32:35.576686 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.576654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:35.636056 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.635976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-serving-cert\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.636238 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.636118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-oauth-config\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.636238 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.636155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-config\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.636238 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.636177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-service-ca\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.636238 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.636191    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-trusted-ca-bundle\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.636393 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.636250    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-oauth-serving-cert\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.636393 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.636321    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5kzs\" (UniqueName: \"kubernetes.io/projected/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-kube-api-access-s5kzs\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.736974 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.736932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-serving-cert\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737053    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-oauth-config\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737101    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-config\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-service-ca\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-trusted-ca-bundle\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-oauth-serving-cert\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737497 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737428    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5kzs\" (UniqueName: \"kubernetes.io/projected/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-kube-api-access-s5kzs\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.737983 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.737921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-service-ca\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.738148 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.738123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-config\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.738219 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.738157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-trusted-ca-bundle\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.738347 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.738322    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-oauth-serving-cert\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.739741 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.739718    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-serving-cert\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.739854 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.739837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-oauth-config\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.745901 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.745880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5kzs\" (UniqueName: \"kubernetes.io/projected/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-kube-api-access-s5kzs\") pod \"console-58fb78d5bd-tdxkb\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") " pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.826228 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.826141    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:35.967614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:35.967450    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58fb78d5bd-tdxkb"]
Jul 29 19:32:35.970366 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:32:35.970334    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d6b99f3_d273_4a12_a0ac_2fe06b74c82b.slice/crio-04c57013564efdffaed0ccafcbbbd144f1fd4c03c821207ee0434344d0aec12e WatchSource:0}: Error finding container 04c57013564efdffaed0ccafcbbbd144f1fd4c03c821207ee0434344d0aec12e: Status 404 returned error can't find the container with id 04c57013564efdffaed0ccafcbbbd144f1fd4c03c821207ee0434344d0aec12e
Jul 29 19:32:36.690078 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:36.690033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58fb78d5bd-tdxkb" event={"ID":"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b","Type":"ContainerStarted","Data":"01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34"}
Jul 29 19:32:36.690253 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:36.690089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58fb78d5bd-tdxkb" event={"ID":"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b","Type":"ContainerStarted","Data":"04c57013564efdffaed0ccafcbbbd144f1fd4c03c821207ee0434344d0aec12e"}
Jul 29 19:32:36.710692 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:36.710631    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58fb78d5bd-tdxkb" podStartSLOduration=1.710615744 podStartE2EDuration="1.710615744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:32:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:32:36.708773216 +0000 UTC m=+178.586627544" watchObservedRunningTime="2026-07-29 19:32:36.710615744 +0000 UTC m=+178.588470072"
Jul 29 19:32:45.826811 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:45.826772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:45.827322 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:45.826840    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:45.831555 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:45.831531    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:46.735688 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:46.735661    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:32:46.798286 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:46.798253    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bcc8fd48-l9d99"]
Jul 29 19:32:46.919271 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:46.919214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:46.974567 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:46.974528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:47.059537 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:47.059451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:32:54.285289 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:32:54.285260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:11.819051 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:11.818989    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8bcc8fd48-l9d99" podUID="27e9a06b-7e29-46dc-8897-662e9d73511d" containerName="console" containerID="cri-o://0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589" gracePeriod=15
Jul 29 19:33:12.052484 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.052454    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:33:12.152267 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152225    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-console-config\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152447 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152270    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-spl5g\" (UniqueName: \"kubernetes.io/projected/27e9a06b-7e29-46dc-8897-662e9d73511d-kube-api-access-spl5g\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152447 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152319    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-service-ca\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152447 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-oauth-serving-cert\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152447 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-serving-cert\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152447 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152446    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-oauth-config\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152712 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152503    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-trusted-ca-bundle\") pod \"27e9a06b-7e29-46dc-8897-662e9d73511d\" (UID: \"27e9a06b-7e29-46dc-8897-662e9d73511d\") "
Jul 29 19:33:12.152771 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-console-config" (OuterVolumeSpecName: "console-config") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:33:12.152842 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:33:12.152842 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.152820    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-service-ca" (OuterVolumeSpecName: "service-ca") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:33:12.153106 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.153082    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:33:12.154751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.154726    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:33:12.154751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.154740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:33:12.154891 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.154810    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27e9a06b-7e29-46dc-8897-662e9d73511d-kube-api-access-spl5g" (OuterVolumeSpecName: "kube-api-access-spl5g") pod "27e9a06b-7e29-46dc-8897-662e9d73511d" (UID: "27e9a06b-7e29-46dc-8897-662e9d73511d"). InnerVolumeSpecName "kube-api-access-spl5g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:33:12.253859 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253813    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-trusted-ca-bundle\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.253859 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253852    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-console-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.253859 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253865    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-spl5g\" (UniqueName: \"kubernetes.io/projected/27e9a06b-7e29-46dc-8897-662e9d73511d-kube-api-access-spl5g\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.254137 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253877    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.254137 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253889    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27e9a06b-7e29-46dc-8897-662e9d73511d-oauth-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.254137 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253901    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.254137 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.253912    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27e9a06b-7e29-46dc-8897-662e9d73511d-console-oauth-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:33:12.649395 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.649352    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:12.838730 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.838696    2580 generic.go:356] "Generic (PLEG): container finished" podID="27e9a06b-7e29-46dc-8897-662e9d73511d" containerID="0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589" exitCode=0
Jul 29 19:33:12.839184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.838744    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bcc8fd48-l9d99"
Jul 29 19:33:12.839184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.838764    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcc8fd48-l9d99" event={"ID":"27e9a06b-7e29-46dc-8897-662e9d73511d","Type":"ContainerDied","Data":"0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589"}
Jul 29 19:33:12.839184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.838808    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bcc8fd48-l9d99" event={"ID":"27e9a06b-7e29-46dc-8897-662e9d73511d","Type":"ContainerDied","Data":"7b27302a6e71a363203b9c2f7f5eb2502170b2b705afa893130ca48b5a239081"}
Jul 29 19:33:12.839184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.838827    2580 scope.go:117] "RemoveContainer" containerID="0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589"
Jul 29 19:33:12.847305 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.847289    2580 scope.go:117] "RemoveContainer" containerID="0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589"
Jul 29 19:33:12.847555 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:33:12.847539    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589\": container with ID starting with 0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589 not found: ID does not exist" containerID="0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589"
Jul 29 19:33:12.847605 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.847564    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589"} err="failed to get container status \"0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589\": rpc error: code = NotFound desc = could not find container \"0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589\": container with ID starting with 0cfc1d0c74a8157b88e91830891d8e7cd7475cd53c57f438a598f4a804b57589 not found: ID does not exist"
Jul 29 19:33:12.860711 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.860685    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bcc8fd48-l9d99"]
Jul 29 19:33:12.866783 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:12.866760    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8bcc8fd48-l9d99"]
Jul 29 19:33:14.715303 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:14.715269    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27e9a06b-7e29-46dc-8897-662e9d73511d" path="/var/lib/kubelet/pods/27e9a06b-7e29-46dc-8897-662e9d73511d/volumes"
Jul 29 19:33:26.717899 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:26.717865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:30.559357 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:30.559327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:30.621326 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:30.621287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:42.116131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:42.116098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:42.171488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:42.171456    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:45.899854 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.899821    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68ffc6c8b4-t7wwh"]
Jul 29 19:33:45.900318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.900181    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27e9a06b-7e29-46dc-8897-662e9d73511d" containerName="console"
Jul 29 19:33:45.900318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.900193    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="27e9a06b-7e29-46dc-8897-662e9d73511d" containerName="console"
Jul 29 19:33:45.900318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.900259    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="27e9a06b-7e29-46dc-8897-662e9d73511d" containerName="console"
Jul 29 19:33:45.902733 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.902717    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:45.919528 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.919496    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffc6c8b4-t7wwh"]
Jul 29 19:33:45.964664 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:45.964634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:46.040984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.040950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-service-ca\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.041155 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.040982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-trusted-ca-bundle\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.041155 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.041063    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-oauth-serving-cert\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.041155 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.041122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-serving-cert\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.041259 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.041171    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-config\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.041292 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.041249    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-oauth-config\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.041292 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.041278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h2g6\" (UniqueName: \"kubernetes.io/projected/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-kube-api-access-4h2g6\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.142601 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-service-ca\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.142800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142610    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-trusted-ca-bundle\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.142800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142641    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-oauth-serving-cert\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.142800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142680    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-serving-cert\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.142800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-config\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.142800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-oauth-config\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.143087 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.142810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h2g6\" (UniqueName: \"kubernetes.io/projected/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-kube-api-access-4h2g6\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.143387 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.143338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-service-ca\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.143679 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.143446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-trusted-ca-bundle\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.143754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.143704    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-config\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.143754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.143716    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-oauth-serving-cert\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.145358 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.145336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-serving-cert\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.145475 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.145454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-oauth-config\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.151213 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.151166    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h2g6\" (UniqueName: \"kubernetes.io/projected/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-kube-api-access-4h2g6\") pod \"console-68ffc6c8b4-t7wwh\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") " pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.212246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.212212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:46.358104 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.358075    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68ffc6c8b4-t7wwh"]
Jul 29 19:33:46.360203 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:33:46.360174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef6d24bd_856b_45ab_9a3d_a496a1983fdc.slice/crio-99900f41791dc07bfaec975c3c42e0395857f51fe1a16b23203e1bfca8190542 WatchSource:0}: Error finding container 99900f41791dc07bfaec975c3c42e0395857f51fe1a16b23203e1bfca8190542: Status 404 returned error can't find the container with id 99900f41791dc07bfaec975c3c42e0395857f51fe1a16b23203e1bfca8190542
Jul 29 19:33:46.982924 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.982890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc6c8b4-t7wwh" event={"ID":"ef6d24bd-856b-45ab-9a3d-a496a1983fdc","Type":"ContainerStarted","Data":"23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a"}
Jul 29 19:33:46.983315 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:46.982942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc6c8b4-t7wwh" event={"ID":"ef6d24bd-856b-45ab-9a3d-a496a1983fdc","Type":"ContainerStarted","Data":"99900f41791dc07bfaec975c3c42e0395857f51fe1a16b23203e1bfca8190542"}
Jul 29 19:33:47.002172 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:47.002104    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68ffc6c8b4-t7wwh" podStartSLOduration=2.002085795 podStartE2EDuration="2.002085795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:33:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:33:47.001387768 +0000 UTC m=+248.879242096" watchObservedRunningTime="2026-07-29 19:33:47.002085795 +0000 UTC m=+248.879940125"
Jul 29 19:33:54.127500 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:54.127460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:54.188279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:54.188247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:56.213119 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:56.213082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:56.213549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:56.213135    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:56.218008 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:56.217971    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:57.029122 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:57.029089    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:33:57.082422 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:57.082392    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58fb78d5bd-tdxkb"]
Jul 29 19:33:57.191888 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:57.191854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:57.246980 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:57.246946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:33:57.301886 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:33:57.301798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:00.075921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:00.075838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:00.435754 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:00.435719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:00.537401 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:00.537366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:00.600717 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:00.600682    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:22.104697 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.104633    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58fb78d5bd-tdxkb" podUID="2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" containerName="console" containerID="cri-o://01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34" gracePeriod=15
Jul 29 19:34:22.351271 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.351239    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:34:22.454689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454652    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-trusted-ca-bundle\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.454689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-serving-cert\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.454948 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454711    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-oauth-config\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.454948 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454735    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s5kzs\" (UniqueName: \"kubernetes.io/projected/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-kube-api-access-s5kzs\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.454948 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-service-ca\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.454948 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-oauth-serving-cert\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.454948 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.454864    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-config\") pod \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\" (UID: \"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b\") "
Jul 29 19:34:22.455319 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.455286    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:34:22.455398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.455306    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-service-ca" (OuterVolumeSpecName: "service-ca") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:34:22.455398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.455315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:34:22.455493 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.455433    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-config" (OuterVolumeSpecName: "console-config") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:34:22.456975 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.456953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:34:22.457080 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.457009    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:34:22.457163 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.457145    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-kube-api-access-s5kzs" (OuterVolumeSpecName: "kube-api-access-s5kzs") pod "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" (UID: "2d6b99f3-d273-4a12-a0ac-2fe06b74c82b"). InnerVolumeSpecName "kube-api-access-s5kzs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:34:22.556032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.555964    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:22.556032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.556028    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-trusted-ca-bundle\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:22.556032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.556039    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:22.556032 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.556048    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-console-oauth-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:22.556298 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.556057    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s5kzs\" (UniqueName: \"kubernetes.io/projected/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-kube-api-access-s5kzs\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:22.556298 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.556066    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:22.556298 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:22.556087    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b-oauth-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:34:23.127773 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.127736    2580 generic.go:356] "Generic (PLEG): container finished" podID="2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" containerID="01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34" exitCode=0
Jul 29 19:34:23.128179 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.127784    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58fb78d5bd-tdxkb"
Jul 29 19:34:23.128179 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.127820    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58fb78d5bd-tdxkb" event={"ID":"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b","Type":"ContainerDied","Data":"01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34"}
Jul 29 19:34:23.128179 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.127874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58fb78d5bd-tdxkb" event={"ID":"2d6b99f3-d273-4a12-a0ac-2fe06b74c82b","Type":"ContainerDied","Data":"04c57013564efdffaed0ccafcbbbd144f1fd4c03c821207ee0434344d0aec12e"}
Jul 29 19:34:23.128179 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.127896    2580 scope.go:117] "RemoveContainer" containerID="01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34"
Jul 29 19:34:23.136458 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.136435    2580 scope.go:117] "RemoveContainer" containerID="01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34"
Jul 29 19:34:23.136730 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:34:23.136709    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34\": container with ID starting with 01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34 not found: ID does not exist" containerID="01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34"
Jul 29 19:34:23.136794 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.136739    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34"} err="failed to get container status \"01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34\": rpc error: code = NotFound desc = could not find container \"01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34\": container with ID starting with 01196e7c9a92c0b327a31b43aad396345910031e423e8065d5c44c449ba9ae34 not found: ID does not exist"
Jul 29 19:34:23.146089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.146062    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58fb78d5bd-tdxkb"]
Jul 29 19:34:23.150815 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:23.150793    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58fb78d5bd-tdxkb"]
Jul 29 19:34:24.714648 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:24.714614    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" path="/var/lib/kubelet/pods/2d6b99f3-d273-4a12-a0ac-2fe06b74c82b/volumes"
Jul 29 19:34:33.538704 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:33.538669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:33.635865 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:33.635833    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:34:38.542501 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:38.542474    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:34:38.555500 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:38.555473    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:34:38.556529 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:38.556505    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:34:38.559189 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:38.559172    2580 kubelet.go:1648] "Image garbage collection succeeded"
Jul 29 19:34:38.564398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:34:38.564375    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:35:00.035050 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:00.034983    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:01.461123 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.461088    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:01.917069 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.917029    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"]
Jul 29 19:35:01.917455 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.917438    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" containerName="console"
Jul 29 19:35:01.917535 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.917458    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" containerName="console"
Jul 29 19:35:01.917592 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.917547    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d6b99f3-d273-4a12-a0ac-2fe06b74c82b" containerName="console"
Jul 29 19:35:01.920389 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.920369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:01.922735 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.922715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-jmnjm\""
Jul 29 19:35:01.922975 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.922951    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Jul 29 19:35:01.923088 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.923011    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:35:01.932764 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.932737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"]
Jul 29 19:35:01.997590 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.997555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52272532-da0c-42e1-8afa-1e759fbbbadd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m6nzr\" (UID: \"52272532-da0c-42e1-8afa-1e759fbbbadd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:01.997590 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:01.997595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m47l8\" (UniqueName: \"kubernetes.io/projected/52272532-da0c-42e1-8afa-1e759fbbbadd-kube-api-access-m47l8\") pod \"cert-manager-operator-controller-manager-55c444bf86-m6nzr\" (UID: \"52272532-da0c-42e1-8afa-1e759fbbbadd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:02.098541 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.098510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52272532-da0c-42e1-8afa-1e759fbbbadd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m6nzr\" (UID: \"52272532-da0c-42e1-8afa-1e759fbbbadd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:02.098541 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.098547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m47l8\" (UniqueName: \"kubernetes.io/projected/52272532-da0c-42e1-8afa-1e759fbbbadd-kube-api-access-m47l8\") pod \"cert-manager-operator-controller-manager-55c444bf86-m6nzr\" (UID: \"52272532-da0c-42e1-8afa-1e759fbbbadd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:02.098897 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.098874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/52272532-da0c-42e1-8afa-1e759fbbbadd-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-m6nzr\" (UID: \"52272532-da0c-42e1-8afa-1e759fbbbadd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:02.113596 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.113565    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m47l8\" (UniqueName: \"kubernetes.io/projected/52272532-da0c-42e1-8afa-1e759fbbbadd-kube-api-access-m47l8\") pod \"cert-manager-operator-controller-manager-55c444bf86-m6nzr\" (UID: \"52272532-da0c-42e1-8afa-1e759fbbbadd\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:02.229945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.229865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"
Jul 29 19:35:02.391509 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.391482    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr"]
Jul 29 19:35:02.394082 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:35:02.394055    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52272532_da0c_42e1_8afa_1e759fbbbadd.slice/crio-84a84894910f93c8043098bc3574e74faec578d7423fc73ce03382b3e4b04f2f WatchSource:0}: Error finding container 84a84894910f93c8043098bc3574e74faec578d7423fc73ce03382b3e4b04f2f: Status 404 returned error can't find the container with id 84a84894910f93c8043098bc3574e74faec578d7423fc73ce03382b3e4b04f2f
Jul 29 19:35:02.396689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:02.396672    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:35:03.298474 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:03.298436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr" event={"ID":"52272532-da0c-42e1-8afa-1e759fbbbadd","Type":"ContainerStarted","Data":"84a84894910f93c8043098bc3574e74faec578d7423fc73ce03382b3e4b04f2f"}
Jul 29 19:35:03.602645 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:03.602558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:03.658692 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:03.658643    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:05.310144 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:05.310105    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr" event={"ID":"52272532-da0c-42e1-8afa-1e759fbbbadd","Type":"ContainerStarted","Data":"794cf64b21748a48239e19952d6db774d864236876843c6b4fe74ed4f8e32e0b"}
Jul 29 19:35:05.331141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:05.331083    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-m6nzr" podStartSLOduration=1.543455828 podStartE2EDuration="4.331064055s" totalImagesPullingTime="2.787608227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:35:05.330127013 +0000 UTC m=+327.207981354" watchObservedRunningTime="2026-07-29 19:35:05.331064055 +0000 UTC m=+327.208918385"
Jul 29 19:35:05.774742 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:05.774711    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:05.972679 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:05.972648    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:06.083189 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.083105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:06.194966 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.194930    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5cm52"]
Jul 29 19:35:06.198440 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.198422    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.200596 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.200566    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 29 19:35:06.200721 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.200569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-t74xm\""
Jul 29 19:35:06.200908 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.200894    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 29 19:35:06.209596 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.209575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5cm52"]
Jul 29 19:35:06.335233 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.335149    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n724h\" (UniqueName: \"kubernetes.io/projected/02368182-456c-4e0a-a824-c7d7ad7b8e98-kube-api-access-n724h\") pod \"cert-manager-webhook-6486ffd997-5cm52\" (UID: \"02368182-456c-4e0a-a824-c7d7ad7b8e98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.335233 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.335196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02368182-456c-4e0a-a824-c7d7ad7b8e98-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5cm52\" (UID: \"02368182-456c-4e0a-a824-c7d7ad7b8e98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.436692 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.436637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n724h\" (UniqueName: \"kubernetes.io/projected/02368182-456c-4e0a-a824-c7d7ad7b8e98-kube-api-access-n724h\") pod \"cert-manager-webhook-6486ffd997-5cm52\" (UID: \"02368182-456c-4e0a-a824-c7d7ad7b8e98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.436692 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.436699    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02368182-456c-4e0a-a824-c7d7ad7b8e98-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5cm52\" (UID: \"02368182-456c-4e0a-a824-c7d7ad7b8e98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.445553 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.445518    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/02368182-456c-4e0a-a824-c7d7ad7b8e98-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-5cm52\" (UID: \"02368182-456c-4e0a-a824-c7d7ad7b8e98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.445881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.445858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n724h\" (UniqueName: \"kubernetes.io/projected/02368182-456c-4e0a-a824-c7d7ad7b8e98-kube-api-access-n724h\") pod \"cert-manager-webhook-6486ffd997-5cm52\" (UID: \"02368182-456c-4e0a-a824-c7d7ad7b8e98\") " pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.515702 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.515661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:06.652201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:06.652166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-5cm52"]
Jul 29 19:35:06.655415 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:35:06.655380    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02368182_456c_4e0a_a824_c7d7ad7b8e98.slice/crio-378fc4e4ed0086cfb1ac1fb193c9d6489a5eafe0545d836e5dcb9a729be6337e WatchSource:0}: Error finding container 378fc4e4ed0086cfb1ac1fb193c9d6489a5eafe0545d836e5dcb9a729be6337e: Status 404 returned error can't find the container with id 378fc4e4ed0086cfb1ac1fb193c9d6489a5eafe0545d836e5dcb9a729be6337e
Jul 29 19:35:07.319758 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:07.319723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52" event={"ID":"02368182-456c-4e0a-a824-c7d7ad7b8e98","Type":"ContainerStarted","Data":"378fc4e4ed0086cfb1ac1fb193c9d6489a5eafe0545d836e5dcb9a729be6337e"}
Jul 29 19:35:08.360882 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:08.360850    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:08.578915 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:08.578886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:10.798061 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:10.797951    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:10.898783 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:10.898744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:11.342008 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:11.341972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52" event={"ID":"02368182-456c-4e0a-a824-c7d7ad7b8e98","Type":"ContainerStarted","Data":"8bb009552a42f5951cebc242b76964ba31c72aef5361c141ee2a9ca32bb332b2"}
Jul 29 19:35:11.342180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:11.342044    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:11.359973 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:11.359922    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52" podStartSLOduration=1.520483013 podStartE2EDuration="5.359907312s" totalImagesPullingTime="3.839424299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:35:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:35:11.358704144 +0000 UTC m=+333.236558493" watchObservedRunningTime="2026-07-29 19:35:11.359907312 +0000 UTC m=+333.237761639"
Jul 29 19:35:13.424968 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:13.424932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:13.483116 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:13.483082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:15.769109 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:15.769078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:17.347646 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.347617    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-5cm52"
Jul 29 19:35:17.407141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.407057    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:17.509565 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.509531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:17.865182 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.865107    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqwkd"]
Jul 29 19:35:17.868537 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.868515    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:17.870250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.870235    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-qqp76\""
Jul 29 19:35:17.878480 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.878457    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqwkd"]
Jul 29 19:35:17.940763 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.940727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1c8c670-873b-43cd-a642-cde83637486e-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqwkd\" (UID: \"f1c8c670-873b-43cd-a642-cde83637486e\") " pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:17.940933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:17.940850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc5fd\" (UniqueName: \"kubernetes.io/projected/f1c8c670-873b-43cd-a642-cde83637486e-kube-api-access-jc5fd\") pod \"cert-manager-55b4d954d5-nqwkd\" (UID: \"f1c8c670-873b-43cd-a642-cde83637486e\") " pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:18.042005 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.041967    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc5fd\" (UniqueName: \"kubernetes.io/projected/f1c8c670-873b-43cd-a642-cde83637486e-kube-api-access-jc5fd\") pod \"cert-manager-55b4d954d5-nqwkd\" (UID: \"f1c8c670-873b-43cd-a642-cde83637486e\") " pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:18.042170 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.042029    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1c8c670-873b-43cd-a642-cde83637486e-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqwkd\" (UID: \"f1c8c670-873b-43cd-a642-cde83637486e\") " pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:18.050847 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.050822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1c8c670-873b-43cd-a642-cde83637486e-bound-sa-token\") pod \"cert-manager-55b4d954d5-nqwkd\" (UID: \"f1c8c670-873b-43cd-a642-cde83637486e\") " pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:18.050981 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.050962    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc5fd\" (UniqueName: \"kubernetes.io/projected/f1c8c670-873b-43cd-a642-cde83637486e-kube-api-access-jc5fd\") pod \"cert-manager-55b4d954d5-nqwkd\" (UID: \"f1c8c670-873b-43cd-a642-cde83637486e\") " pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:18.177987 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.177954    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-nqwkd"
Jul 29 19:35:18.319898 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.319875    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-nqwkd"]
Jul 29 19:35:18.321960 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:35:18.321934    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1c8c670_873b_43cd_a642_cde83637486e.slice/crio-ef57fb69b3b0abdee381a650f8346f3089344b97a746cf361084266ed5fe9d8f WatchSource:0}: Error finding container ef57fb69b3b0abdee381a650f8346f3089344b97a746cf361084266ed5fe9d8f: Status 404 returned error can't find the container with id ef57fb69b3b0abdee381a650f8346f3089344b97a746cf361084266ed5fe9d8f
Jul 29 19:35:18.374051 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:18.374023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nqwkd" event={"ID":"f1c8c670-873b-43cd-a642-cde83637486e","Type":"ContainerStarted","Data":"ef57fb69b3b0abdee381a650f8346f3089344b97a746cf361084266ed5fe9d8f"}
Jul 29 19:35:19.379895 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.379850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-nqwkd" event={"ID":"f1c8c670-873b-43cd-a642-cde83637486e","Type":"ContainerStarted","Data":"7e2e5c69df447fc8624848ac6f03ee10fd92fb0a888214afdd4dffe8b2cb5678"}
Jul 29 19:35:19.400162 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.400112    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-nqwkd" podStartSLOduration=2.40009739 podStartE2EDuration="2.40009739s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:35:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:35:19.399027059 +0000 UTC m=+341.276881473" watchObservedRunningTime="2026-07-29 19:35:19.40009739 +0000 UTC m=+341.277951718"
Jul 29 19:35:19.425341 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.425302    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"]
Jul 29 19:35:19.428583 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.428566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.430757 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.430738    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-t844n\""
Jul 29 19:35:19.431015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.430985    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 29 19:35:19.431108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.431028    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 29 19:35:19.444212 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.444166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"]
Jul 29 19:35:19.556450 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.556415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9d37d8b-f5f9-4265-828d-dec5586cf62a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kcpzv\" (UID: \"b9d37d8b-f5f9-4265-828d-dec5586cf62a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.556606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.556475    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbspf\" (UniqueName: \"kubernetes.io/projected/b9d37d8b-f5f9-4265-828d-dec5586cf62a-kube-api-access-jbspf\") pod \"openshift-lws-operator-fd8ccff4c-kcpzv\" (UID: \"b9d37d8b-f5f9-4265-828d-dec5586cf62a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.657687 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.657605    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9d37d8b-f5f9-4265-828d-dec5586cf62a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kcpzv\" (UID: \"b9d37d8b-f5f9-4265-828d-dec5586cf62a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.657687 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.657652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbspf\" (UniqueName: \"kubernetes.io/projected/b9d37d8b-f5f9-4265-828d-dec5586cf62a-kube-api-access-jbspf\") pod \"openshift-lws-operator-fd8ccff4c-kcpzv\" (UID: \"b9d37d8b-f5f9-4265-828d-dec5586cf62a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.658064 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.658043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b9d37d8b-f5f9-4265-828d-dec5586cf62a-tmp\") pod \"openshift-lws-operator-fd8ccff4c-kcpzv\" (UID: \"b9d37d8b-f5f9-4265-828d-dec5586cf62a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.666269 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.666250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbspf\" (UniqueName: \"kubernetes.io/projected/b9d37d8b-f5f9-4265-828d-dec5586cf62a-kube-api-access-jbspf\") pod \"openshift-lws-operator-fd8ccff4c-kcpzv\" (UID: \"b9d37d8b-f5f9-4265-828d-dec5586cf62a\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.739698 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.739663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"
Jul 29 19:35:19.872836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:19.872810    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv"]
Jul 29 19:35:19.874852 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:35:19.874814    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb9d37d8b_f5f9_4265_828d_dec5586cf62a.slice/crio-698dfe8e966344cd3ea50230427f60ad87f72da98c4787e9fac45fbc83690864 WatchSource:0}: Error finding container 698dfe8e966344cd3ea50230427f60ad87f72da98c4787e9fac45fbc83690864: Status 404 returned error can't find the container with id 698dfe8e966344cd3ea50230427f60ad87f72da98c4787e9fac45fbc83690864
Jul 29 19:35:20.385563 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:20.385525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv" event={"ID":"b9d37d8b-f5f9-4265-828d-dec5586cf62a","Type":"ContainerStarted","Data":"698dfe8e966344cd3ea50230427f60ad87f72da98c4787e9fac45fbc83690864"}
Jul 29 19:35:23.402119 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:23.402081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv" event={"ID":"b9d37d8b-f5f9-4265-828d-dec5586cf62a","Type":"ContainerStarted","Data":"5e1dd8876695774069b8c724391d7f04e70854db0116d7ebd34a3cb652244968"}
Jul 29 19:35:23.419854 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:23.419798    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-kcpzv" podStartSLOduration=1.377087091 podStartE2EDuration="4.419778186s" totalImagesPullingTime="3.042691095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:35:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:35:23.419546995 +0000 UTC m=+345.297401324" watchObservedRunningTime="2026-07-29 19:35:23.419778186 +0000 UTC m=+345.297632515"
Jul 29 19:35:24.088945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:24.088909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:24.190248 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:24.190212    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:25.811403 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:25.811371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:38.802467 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:38.802376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:38.902027 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:38.901975    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:39.424024 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.423964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:39.663250 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.663196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:39.769414 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.769324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:39.811143 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.811102    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"]
Jul 29 19:35:39.858553 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.858525    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"]
Jul 29 19:35:39.858703 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.858637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:39.860775 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.860650    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 29 19:35:39.860775 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.860733    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 29 19:35:39.860775 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.860720    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 29 19:35:39.860775 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.860740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 29 19:35:39.861131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.860902    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-2vqxw\""
Jul 29 19:35:39.922617 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.922585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:39.930448 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.930415    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-tmp\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:39.930589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.930469    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-manifests\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:39.930589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.930491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:39.930589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.930513    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qn7d\" (UniqueName: \"kubernetes.io/projected/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-kube-api-access-5qn7d\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:39.930589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:39.930534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.031670 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.031587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-manifests\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.031670 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.031625    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.031670 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.031651    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qn7d\" (UniqueName: \"kubernetes.io/projected/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-kube-api-access-5qn7d\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.031890 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.031683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.031890 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.031753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-tmp\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.032127 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.032102    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-manifests\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.034241 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.034215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-tmp\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.034385 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.034346    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.034532 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.034507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-webhook-cert\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.042133 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.042108    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qn7d\" (UniqueName: \"kubernetes.io/projected/e5d77ff4-9f8b-4a93-91c9-f1b30213a39c-kube-api-access-5qn7d\") pod \"opendatahub-operator-controller-manager-5d75b7b96-q7mlp\" (UID: \"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c\") " pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.072117 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.072071    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:40.168186 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.168151    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:40.170097 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.170070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:40.246889 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.246837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:40.320867 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.320744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:40.331573 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.331538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"]
Jul 29 19:35:40.335643 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:35:40.335615    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d77ff4_9f8b_4a93_91c9_f1b30213a39c.slice/crio-1b9c459130f46b94292db399a835be93ba44e25b675636c1e4ebaf4e1f4876cc WatchSource:0}: Error finding container 1b9c459130f46b94292db399a835be93ba44e25b675636c1e4ebaf4e1f4876cc: Status 404 returned error can't find the container with id 1b9c459130f46b94292db399a835be93ba44e25b675636c1e4ebaf4e1f4876cc
Jul 29 19:35:40.421458 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.421426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:40.480062 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:40.480026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp" event={"ID":"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c","Type":"ContainerStarted","Data":"1b9c459130f46b94292db399a835be93ba44e25b675636c1e4ebaf4e1f4876cc"}
Jul 29 19:35:42.160254 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:42.160215    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:42.265236 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:42.265183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:44.502010 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:44.501956    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp" event={"ID":"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c","Type":"ContainerStarted","Data":"95138adc0452625b9bfef128ac2b46179cf8c8c0629fcbd63567e75c8459788e"}
Jul 29 19:35:45.152771 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:35:45.152736    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5d77ff4_9f8b_4a93_91c9_f1b30213a39c.slice/crio-conmon-95138adc0452625b9bfef128ac2b46179cf8c8c0629fcbd63567e75c8459788e.scope\": RecentStats: unable to find data in memory cache]"
Jul 29 19:35:45.508015 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:45.507959    2580 generic.go:356] "Generic (PLEG): container finished" podID="e5d77ff4-9f8b-4a93-91c9-f1b30213a39c" containerID="95138adc0452625b9bfef128ac2b46179cf8c8c0629fcbd63567e75c8459788e" exitCode=0
Jul 29 19:35:45.508405 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:45.508034    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp" event={"ID":"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c","Type":"ContainerDied","Data":"95138adc0452625b9bfef128ac2b46179cf8c8c0629fcbd63567e75c8459788e"}
Jul 29 19:35:46.515112 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:46.515074    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp" event={"ID":"e5d77ff4-9f8b-4a93-91c9-f1b30213a39c","Type":"ContainerStarted","Data":"6ebd3134fc79ca7e884b9fc9ac0e5775d535214213408b95be7f76953e7c2053"}
Jul 29 19:35:46.515840 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:46.515212    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:46.537386 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:46.537322    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp" podStartSLOduration=2.500305916 podStartE2EDuration="7.537296608s" totalImagesPullingTime="3.940668321s" totalInitContainerRuntime="1.096322371s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:35:39 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:35:46.536543438 +0000 UTC m=+368.414397766" watchObservedRunningTime="2026-07-29 19:35:46.537296608 +0000 UTC m=+368.415150937"
Jul 29 19:35:50.968116 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:50.968084    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:51.074210 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:51.074174    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:57.523130 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:57.523100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5d75b7b96-q7mlp"
Jul 29 19:35:59.381470 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.381430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:59.488694 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.488660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:59.632244 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.632157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:35:59.890022 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.889918    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"]
Jul 29 19:35:59.893639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.893614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:35:59.897159 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.897126    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 29 19:35:59.897439 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.897414    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j8k7c\""
Jul 29 19:35:59.898077 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.898056    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 29 19:35:59.908242 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.908218    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"]
Jul 29 19:35:59.985158 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:35:59.985116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:00.013426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.013389    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-tls-certs\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.013619 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.013437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbgqv\" (UniqueName: \"kubernetes.io/projected/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-kube-api-access-fbgqv\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.013619 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.013550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-tmp\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.069381 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.069348    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8t7ch"]
Jul 29 19:36:00.072715 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.072694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.074652 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.074632    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 29 19:36:00.074838 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.074750    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-tfq8j\""
Jul 29 19:36:00.090549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.090526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8t7ch"]
Jul 29 19:36:00.114451 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.114416    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-tmp\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.114632 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.114473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-tls-certs\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.114632 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.114505    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbgqv\" (UniqueName: \"kubernetes.io/projected/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-kube-api-access-fbgqv\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.117074 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.117043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-tmp\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.117267 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.117249    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-tls-certs\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.126286 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.126264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbgqv\" (UniqueName: \"kubernetes.io/projected/fb3bfa3a-a3e6-4290-b230-5a906f026bcb-kube-api-access-fbgqv\") pod \"kube-auth-proxy-6cc65fd5df-xng9r\" (UID: \"fb3bfa3a-a3e6-4290-b230-5a906f026bcb\") " pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.127269 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.127246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:00.204859 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.204763    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"
Jul 29 19:36:00.215394 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.215349    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5kkr\" (UniqueName: \"kubernetes.io/projected/f00a6dce-0179-4df2-9b80-8d22df1b29b7-kube-api-access-w5kkr\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.215549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.215402    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a6dce-0179-4df2-9b80-8d22df1b29b7-cert\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.225076 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.225041    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:00.316917 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.316802    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5kkr\" (UniqueName: \"kubernetes.io/projected/f00a6dce-0179-4df2-9b80-8d22df1b29b7-kube-api-access-w5kkr\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.316917 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.316847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a6dce-0179-4df2-9b80-8d22df1b29b7-cert\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.317183 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:36:00.317031    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 29 19:36:00.317183 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:36:00.317094    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f00a6dce-0179-4df2-9b80-8d22df1b29b7-cert podName:f00a6dce-0179-4df2-9b80-8d22df1b29b7 nodeName:}" failed. No retries permitted until 2026-07-29 19:36:00.817074076 +0000 UTC m=+382.694928397 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f00a6dce-0179-4df2-9b80-8d22df1b29b7-cert") pod "odh-model-controller-7998c67dc4-8t7ch" (UID: "f00a6dce-0179-4df2-9b80-8d22df1b29b7") : secret "odh-model-controller-webhook-cert" not found
Jul 29 19:36:00.330524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.330496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5kkr\" (UniqueName: \"kubernetes.io/projected/f00a6dce-0179-4df2-9b80-8d22df1b29b7-kube-api-access-w5kkr\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.356373 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.354979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r"]
Jul 29 19:36:00.358447 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:36:00.358412    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3bfa3a_a3e6_4290_b230_5a906f026bcb.slice/crio-49384e191ed38d8b72644624d59d11c9388c663d414adcd91a13b85dafbe6f7d WatchSource:0}: Error finding container 49384e191ed38d8b72644624d59d11c9388c663d414adcd91a13b85dafbe6f7d: Status 404 returned error can't find the container with id 49384e191ed38d8b72644624d59d11c9388c663d414adcd91a13b85dafbe6f7d
Jul 29 19:36:00.371639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.371608    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:00.580322 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.580235    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r" event={"ID":"fb3bfa3a-a3e6-4290-b230-5a906f026bcb","Type":"ContainerStarted","Data":"49384e191ed38d8b72644624d59d11c9388c663d414adcd91a13b85dafbe6f7d"}
Jul 29 19:36:00.822609 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.822567    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a6dce-0179-4df2-9b80-8d22df1b29b7-cert\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.825318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.825292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f00a6dce-0179-4df2-9b80-8d22df1b29b7-cert\") pod \"odh-model-controller-7998c67dc4-8t7ch\" (UID: \"f00a6dce-0179-4df2-9b80-8d22df1b29b7\") " pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:00.854703 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.854600    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:00.982965 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:00.982926    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:01.121391 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.121364    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-8t7ch"]
Jul 29 19:36:01.123071 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:36:01.123046    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf00a6dce_0179_4df2_9b80_8d22df1b29b7.slice/crio-6999feba244c297ec2eccfb312de699af87b42f1dde947db06b33ca152e5e1d8 WatchSource:0}: Error finding container 6999feba244c297ec2eccfb312de699af87b42f1dde947db06b33ca152e5e1d8: Status 404 returned error can't find the container with id 6999feba244c297ec2eccfb312de699af87b42f1dde947db06b33ca152e5e1d8
Jul 29 19:36:01.213030 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.212980    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:01.221327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.221293    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"]
Jul 29 19:36:01.225823 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.225801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.229240 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.229219    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 29 19:36:01.229920 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.229892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 29 19:36:01.229920 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.229906    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kgj7d\""
Jul 29 19:36:01.230191 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.230176    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:36:01.230360 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.230345    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 29 19:36:01.245244 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.245218    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"]
Jul 29 19:36:01.311644 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.311527    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:01.327543 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327499    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.327744 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62r7r\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-kube-api-access-62r7r\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.327744 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327676    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.327744 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec18157f-a362-4be0-a7c7-9eaed592cd08-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.327888 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.327888 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.327988 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.327911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429158 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec18157f-a362-4be0-a7c7-9eaed592cd08-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429183    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62r7r\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-kube-api-access-62r7r\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.429830 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.429792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.431912 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.431880    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.432030 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.431886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.432030 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.431979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.432030 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.431983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec18157f-a362-4be0-a7c7-9eaed592cd08-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.437388 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.437367    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62r7r\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-kube-api-access-62r7r\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.437463 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.437433    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-jkd8z\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.536084 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.536042    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:01.585952 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.585768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch" event={"ID":"f00a6dce-0179-4df2-9b80-8d22df1b29b7","Type":"ContainerStarted","Data":"6999feba244c297ec2eccfb312de699af87b42f1dde947db06b33ca152e5e1d8"}
Jul 29 19:36:01.648465 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.648427    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:01.683119 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:01.683094    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"]
Jul 29 19:36:01.685659 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:36:01.685566    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec18157f_a362_4be0_a7c7_9eaed592cd08.slice/crio-306e25f8107859739cad3a86f5da7866a7e044ff642171809ddc248833d72b5c WatchSource:0}: Error finding container 306e25f8107859739cad3a86f5da7866a7e044ff642171809ddc248833d72b5c: Status 404 returned error can't find the container with id 306e25f8107859739cad3a86f5da7866a7e044ff642171809ddc248833d72b5c
Jul 29 19:36:02.284085 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:02.284014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:02.598437 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:02.598351    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" event={"ID":"ec18157f-a362-4be0-a7c7-9eaed592cd08","Type":"ContainerStarted","Data":"306e25f8107859739cad3a86f5da7866a7e044ff642171809ddc248833d72b5c"}
Jul 29 19:36:05.851080 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:05.851040    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:36:05.851362 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:05.851123    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:36:06.623909 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.623877    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch" event={"ID":"f00a6dce-0179-4df2-9b80-8d22df1b29b7","Type":"ContainerStarted","Data":"a746bd4cb1e5f0b659f96f1b0a3b814c8ddd87ec37c42746398f83df05956294"}
Jul 29 19:36:06.624128 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.623967    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:06.625416 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.625382    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r" event={"ID":"fb3bfa3a-a3e6-4290-b230-5a906f026bcb","Type":"ContainerStarted","Data":"eda00e2f3c578e3ba38a9bacccacc1807a781d304cef9812be70dad84367e6b4"}
Jul 29 19:36:06.626879 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.626858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" event={"ID":"ec18157f-a362-4be0-a7c7-9eaed592cd08","Type":"ContainerStarted","Data":"ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020"}
Jul 29 19:36:06.626988 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.626897    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:06.628869 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.628841    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-jkd8z container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 29 19:36:06.628963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.628901    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" podUID="ec18157f-a362-4be0-a7c7-9eaed592cd08" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 29 19:36:06.642164 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.642106    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch" podStartSLOduration=1.918302207 podStartE2EDuration="6.64208569s" totalImagesPullingTime="4.723783483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:36:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:36:06.640555325 +0000 UTC m=+388.518409653" watchObservedRunningTime="2026-07-29 19:36:06.64208569 +0000 UTC m=+388.519940020"
Jul 29 19:36:06.659954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.659898    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6cc65fd5df-xng9r" podStartSLOduration=2.164720982 podStartE2EDuration="7.659884433s" totalImagesPullingTime="5.495163451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:35:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:36:06.657987026 +0000 UTC m=+388.535841356" watchObservedRunningTime="2026-07-29 19:36:06.659884433 +0000 UTC m=+388.537738760"
Jul 29 19:36:06.680792 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.680732    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" podStartSLOduration=1.517961267 podStartE2EDuration="5.680708527s" totalImagesPullingTime="4.16274726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:36:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:36:06.680316811 +0000 UTC m=+388.558171140" watchObservedRunningTime="2026-07-29 19:36:06.680708527 +0000 UTC m=+388.558562856"
Jul 29 19:36:06.794622 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.794580    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:06.901064 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:06.901032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:07.632291 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:07.632259    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:08.019784 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:08.019701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:08.135339 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:08.135308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:10.253370 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.253324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:10.335223 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.335187    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"]
Jul 29 19:36:10.338927 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.338900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:10.339748 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.339721    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"]
Jul 29 19:36:10.342523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.342503    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-99wgz\""
Jul 29 19:36:10.342766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.342748    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 29 19:36:10.343482 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.343461    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.345074 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.345053    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 29 19:36:10.345560 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.345542    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-wkcht\""
Jul 29 19:36:10.354273 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.354250    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"]
Jul 29 19:36:10.360377 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.360357    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"]
Jul 29 19:36:10.388086 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.388045    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:10.422266 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.422235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796bc525-917d-4964-9056-4849fa4a9fbf-cert\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:10.422443 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.422294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nbkd\" (UniqueName: \"kubernetes.io/projected/796bc525-917d-4964-9056-4849fa4a9fbf-kube-api-access-9nbkd\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:10.523676 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.523589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhwfg\" (UniqueName: \"kubernetes.io/projected/a85212a3-270a-4dea-9207-98b2ffd69cd0-kube-api-access-mhwfg\") pod \"llmisvc-controller-manager-777c6bd7d5-w9pl6\" (UID: \"a85212a3-270a-4dea-9207-98b2ffd69cd0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.523676 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.523646    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796bc525-917d-4964-9056-4849fa4a9fbf-cert\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:10.523875 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.523749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nbkd\" (UniqueName: \"kubernetes.io/projected/796bc525-917d-4964-9056-4849fa4a9fbf-kube-api-access-9nbkd\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:10.523875 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:36:10.523769    2580 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Jul 29 19:36:10.523875 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.523802    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a85212a3-270a-4dea-9207-98b2ffd69cd0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w9pl6\" (UID: \"a85212a3-270a-4dea-9207-98b2ffd69cd0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.523875 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:36:10.523824    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/796bc525-917d-4964-9056-4849fa4a9fbf-cert podName:796bc525-917d-4964-9056-4849fa4a9fbf nodeName:}" failed. No retries permitted until 2026-07-29 19:36:11.023808495 +0000 UTC m=+392.901662803 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/796bc525-917d-4964-9056-4849fa4a9fbf-cert") pod "kserve-controller-manager-d6f4b65b8-vslbl" (UID: "796bc525-917d-4964-9056-4849fa4a9fbf") : secret "kserve-webhook-server-cert" not found
Jul 29 19:36:10.537317 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.537294    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nbkd\" (UniqueName: \"kubernetes.io/projected/796bc525-917d-4964-9056-4849fa4a9fbf-kube-api-access-9nbkd\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:10.575126 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.575094    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:10.624353 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.624319    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhwfg\" (UniqueName: \"kubernetes.io/projected/a85212a3-270a-4dea-9207-98b2ffd69cd0-kube-api-access-mhwfg\") pod \"llmisvc-controller-manager-777c6bd7d5-w9pl6\" (UID: \"a85212a3-270a-4dea-9207-98b2ffd69cd0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.624538 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.624474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a85212a3-270a-4dea-9207-98b2ffd69cd0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w9pl6\" (UID: \"a85212a3-270a-4dea-9207-98b2ffd69cd0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.627192 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.627169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a85212a3-270a-4dea-9207-98b2ffd69cd0-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-w9pl6\" (UID: \"a85212a3-270a-4dea-9207-98b2ffd69cd0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.636318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.636289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhwfg\" (UniqueName: \"kubernetes.io/projected/a85212a3-270a-4dea-9207-98b2ffd69cd0-kube-api-access-mhwfg\") pod \"llmisvc-controller-manager-777c6bd7d5-w9pl6\" (UID: \"a85212a3-270a-4dea-9207-98b2ffd69cd0\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.660928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.660896    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:10.773253 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.773221    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:10.820297 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.820216    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"]
Jul 29 19:36:10.823501 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:36:10.823477    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda85212a3_270a_4dea_9207_98b2ffd69cd0.slice/crio-e5f3c66d844f4e212a423bf0fc68a2a7308292cd0983a692e777b6a8b7173928 WatchSource:0}: Error finding container e5f3c66d844f4e212a423bf0fc68a2a7308292cd0983a692e777b6a8b7173928: Status 404 returned error can't find the container with id e5f3c66d844f4e212a423bf0fc68a2a7308292cd0983a692e777b6a8b7173928
Jul 29 19:36:10.863169 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:10.863142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:11.029424 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:11.029390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796bc525-917d-4964-9056-4849fa4a9fbf-cert\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:11.031867 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:11.031847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/796bc525-917d-4964-9056-4849fa4a9fbf-cert\") pod \"kserve-controller-manager-d6f4b65b8-vslbl\" (UID: \"796bc525-917d-4964-9056-4849fa4a9fbf\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:11.253398 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:11.253363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:11.394899 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:11.394872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"]
Jul 29 19:36:11.419436 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:36:11.419410    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod796bc525_917d_4964_9056_4849fa4a9fbf.slice/crio-af19a3ff13727cd4e8e3b385fdc94e5105ca4529d4b4ae476811d34c154f13de WatchSource:0}: Error finding container af19a3ff13727cd4e8e3b385fdc94e5105ca4529d4b4ae476811d34c154f13de: Status 404 returned error can't find the container with id af19a3ff13727cd4e8e3b385fdc94e5105ca4529d4b4ae476811d34c154f13de
Jul 29 19:36:11.651187 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:11.651143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl" event={"ID":"796bc525-917d-4964-9056-4849fa4a9fbf","Type":"ContainerStarted","Data":"af19a3ff13727cd4e8e3b385fdc94e5105ca4529d4b4ae476811d34c154f13de"}
Jul 29 19:36:11.652366 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:11.652338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6" event={"ID":"a85212a3-270a-4dea-9207-98b2ffd69cd0","Type":"ContainerStarted","Data":"e5f3c66d844f4e212a423bf0fc68a2a7308292cd0983a692e777b6a8b7173928"}
Jul 29 19:36:12.845033 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:12.844987    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"]
Jul 29 19:36:12.845507 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:12.845215    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" podUID="ec18157f-a362-4be0-a7c7-9eaed592cd08" containerName="discovery" containerID="cri-o://ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020" gracePeriod=30
Jul 29 19:36:13.138492 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.138462    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:13.250836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.250795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-token\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.250836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.250843    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-cacerts\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.251108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.250875    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-dns-cert\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.251108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.250989    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-ca-configmap\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.251108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.251053    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec18157f-a362-4be0-a7c7-9eaed592cd08-local-certs\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.251108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.251096    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62r7r\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-kube-api-access-62r7r\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.251335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.251136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-kubeconfig\") pod \"ec18157f-a362-4be0-a7c7-9eaed592cd08\" (UID: \"ec18157f-a362-4be0-a7c7-9eaed592cd08\") "
Jul 29 19:36:13.251921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.251885    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:36:13.254268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.254222    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:36:13.254431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.254389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-token" (OuterVolumeSpecName: "istio-token") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:36:13.254711 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.254685    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-cacerts" (OuterVolumeSpecName: "cacerts") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:36:13.254963 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.254933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:36:13.255112 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.255081    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-kube-api-access-62r7r" (OuterVolumeSpecName: "kube-api-access-62r7r") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "kube-api-access-62r7r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:36:13.255741 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.255722    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ec18157f-a362-4be0-a7c7-9eaed592cd08-local-certs" (OuterVolumeSpecName: "local-certs") pod "ec18157f-a362-4be0-a7c7-9eaed592cd08" (UID: "ec18157f-a362-4be0-a7c7-9eaed592cd08"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:36:13.352466 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352418    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-token\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.352466 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352460    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-cacerts\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.352466 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352473    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-dns-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.352749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352486    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-csr-ca-configmap\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.352749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352501    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/ec18157f-a362-4be0-a7c7-9eaed592cd08-local-certs\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.352749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352513    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62r7r\" (UniqueName: \"kubernetes.io/projected/ec18157f-a362-4be0-a7c7-9eaed592cd08-kube-api-access-62r7r\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.352749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.352523    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ec18157f-a362-4be0-a7c7-9eaed592cd08-istio-kubeconfig\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:36:13.665305 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.665266    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec18157f-a362-4be0-a7c7-9eaed592cd08" containerID="ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020" exitCode=0
Jul 29 19:36:13.665502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.665316    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"
Jul 29 19:36:13.665502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.665315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" event={"ID":"ec18157f-a362-4be0-a7c7-9eaed592cd08","Type":"ContainerDied","Data":"ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020"}
Jul 29 19:36:13.665502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.665470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z" event={"ID":"ec18157f-a362-4be0-a7c7-9eaed592cd08","Type":"ContainerDied","Data":"306e25f8107859739cad3a86f5da7866a7e044ff642171809ddc248833d72b5c"}
Jul 29 19:36:13.665502 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.665498    2580 scope.go:117] "RemoveContainer" containerID="ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020"
Jul 29 19:36:13.695571 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.695533    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"]
Jul 29 19:36:13.703153 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:13.703113    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-jkd8z"]
Jul 29 19:36:14.418619 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.418447    2580 scope.go:117] "RemoveContainer" containerID="ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020"
Jul 29 19:36:14.418910 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:36:14.418836    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020\": container with ID starting with ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020 not found: ID does not exist" containerID="ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020"
Jul 29 19:36:14.418910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.418876    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020"} err="failed to get container status \"ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020\": rpc error: code = NotFound desc = could not find container \"ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020\": container with ID starting with ee70d282b9e8383c9f5dd8260c31a9b0f8880a21204a9ec8dee406a135c28020 not found: ID does not exist"
Jul 29 19:36:14.671691 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.671587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6" event={"ID":"a85212a3-270a-4dea-9207-98b2ffd69cd0","Type":"ContainerStarted","Data":"d345dff754a289cc0c2ef806b1d2ff6b65a8363b5d91c18f82346303d81b21d6"}
Jul 29 19:36:14.671878 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.671749    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:14.673989 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.673955    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl" event={"ID":"796bc525-917d-4964-9056-4849fa4a9fbf","Type":"ContainerStarted","Data":"9185da01049646aaff57c0861c4e4d7557eec1e5bdd85cb74ca7c94c0a252f6b"}
Jul 29 19:36:14.674144 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.674066    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:36:14.690275 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.690208    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6" podStartSLOduration=1.0836089 podStartE2EDuration="4.690188155s" totalImagesPullingTime="3.606579255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:36:14.689933191 +0000 UTC m=+396.567787521" watchObservedRunningTime="2026-07-29 19:36:14.690188155 +0000 UTC m=+396.568042486"
Jul 29 19:36:14.708368 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.708319    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl" podStartSLOduration=1.70186129 podStartE2EDuration="4.708302953s" totalImagesPullingTime="3.006441663s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:36:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:36:14.707258081 +0000 UTC m=+396.585112410" watchObservedRunningTime="2026-07-29 19:36:14.708302953 +0000 UTC m=+396.586157281"
Jul 29 19:36:14.716789 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:14.716763    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec18157f-a362-4be0-a7c7-9eaed592cd08" path="/var/lib/kubelet/pods/ec18157f-a362-4be0-a7c7-9eaed592cd08/volumes"
Jul 29 19:36:17.634537 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:17.634501    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-8t7ch"
Jul 29 19:36:39.747844 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:39.747805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:39.842808 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:39.842774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:36:45.681421 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:45.681390    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-w9pl6"
Jul 29 19:36:45.684563 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:36:45.684539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-vslbl"
Jul 29 19:37:00.787898 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.787816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:00.882781 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.882747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:00.919700 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.919662    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sdkt4"]
Jul 29 19:37:00.920051 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.920038    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec18157f-a362-4be0-a7c7-9eaed592cd08" containerName="discovery"
Jul 29 19:37:00.920111 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.920052    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec18157f-a362-4be0-a7c7-9eaed592cd08" containerName="discovery"
Jul 29 19:37:00.920147 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.920123    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec18157f-a362-4be0-a7c7-9eaed592cd08" containerName="discovery"
Jul 29 19:37:00.922197 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.922176    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:00.923799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.923770    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 29 19:37:00.924097 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.924078    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 29 19:37:00.924179 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.924108    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lc75p\""
Jul 29 19:37:00.931415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.931389    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sdkt4"]
Jul 29 19:37:00.977743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:00.977705    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:01.076719 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.076631    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56kjq\" (UniqueName: \"kubernetes.io/projected/94b19775-6e4c-48d8-8f3f-5581ea73fc21-kube-api-access-56kjq\") pod \"kuadrant-operator-catalog-sdkt4\" (UID: \"94b19775-6e4c-48d8-8f3f-5581ea73fc21\") " pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:01.177372 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.177335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56kjq\" (UniqueName: \"kubernetes.io/projected/94b19775-6e4c-48d8-8f3f-5581ea73fc21-kube-api-access-56kjq\") pod \"kuadrant-operator-catalog-sdkt4\" (UID: \"94b19775-6e4c-48d8-8f3f-5581ea73fc21\") " pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:01.192457 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.192430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-56kjq\" (UniqueName: \"kubernetes.io/projected/94b19775-6e4c-48d8-8f3f-5581ea73fc21-kube-api-access-56kjq\") pod \"kuadrant-operator-catalog-sdkt4\" (UID: \"94b19775-6e4c-48d8-8f3f-5581ea73fc21\") " pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:01.233805 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.233772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:01.273564 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.273531    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sdkt4"]
Jul 29 19:37:01.369844 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.369816    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sdkt4"]
Jul 29 19:37:01.372605 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:01.372579    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94b19775_6e4c_48d8_8f3f_5581ea73fc21.slice/crio-522224f32ff04e81c69b490e96319e51ba0e73c7d17e0b8f5b9e4a69c5814535 WatchSource:0}: Error finding container 522224f32ff04e81c69b490e96319e51ba0e73c7d17e0b8f5b9e4a69c5814535: Status 404 returned error can't find the container with id 522224f32ff04e81c69b490e96319e51ba0e73c7d17e0b8f5b9e4a69c5814535
Jul 29 19:37:01.484431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.484387    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-54w2z"]
Jul 29 19:37:01.487364 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.487347    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:01.496339 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.496315    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-54w2z"]
Jul 29 19:37:01.581213 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.581176    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfp57\" (UniqueName: \"kubernetes.io/projected/7aeac165-0eb7-4d3a-b6c6-a5e820016ae3-kube-api-access-qfp57\") pod \"kuadrant-operator-catalog-54w2z\" (UID: \"7aeac165-0eb7-4d3a-b6c6-a5e820016ae3\") " pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:01.681911 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.681867    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfp57\" (UniqueName: \"kubernetes.io/projected/7aeac165-0eb7-4d3a-b6c6-a5e820016ae3-kube-api-access-qfp57\") pod \"kuadrant-operator-catalog-54w2z\" (UID: \"7aeac165-0eb7-4d3a-b6c6-a5e820016ae3\") " pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:01.690207 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.690173    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfp57\" (UniqueName: \"kubernetes.io/projected/7aeac165-0eb7-4d3a-b6c6-a5e820016ae3-kube-api-access-qfp57\") pod \"kuadrant-operator-catalog-54w2z\" (UID: \"7aeac165-0eb7-4d3a-b6c6-a5e820016ae3\") " pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:01.713940 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.713904    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:01.797387 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.797350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:01.911050 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.911009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4" event={"ID":"94b19775-6e4c-48d8-8f3f-5581ea73fc21","Type":"ContainerStarted","Data":"522224f32ff04e81c69b490e96319e51ba0e73c7d17e0b8f5b9e4a69c5814535"}
Jul 29 19:37:01.915804 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.915698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:01.938111 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:01.938085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-54w2z"]
Jul 29 19:37:01.940273 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:01.940249    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aeac165_0eb7_4d3a_b6c6_a5e820016ae3.slice/crio-280eba1c51c8445e7616be819b73724c8fc28450bef506effcf1b4166a2c4f27 WatchSource:0}: Error finding container 280eba1c51c8445e7616be819b73724c8fc28450bef506effcf1b4166a2c4f27: Status 404 returned error can't find the container with id 280eba1c51c8445e7616be819b73724c8fc28450bef506effcf1b4166a2c4f27
Jul 29 19:37:02.918347 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:02.918315    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-54w2z" event={"ID":"7aeac165-0eb7-4d3a-b6c6-a5e820016ae3","Type":"ContainerStarted","Data":"280eba1c51c8445e7616be819b73724c8fc28450bef506effcf1b4166a2c4f27"}
Jul 29 19:37:03.924434 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:03.924391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4" event={"ID":"94b19775-6e4c-48d8-8f3f-5581ea73fc21","Type":"ContainerStarted","Data":"7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219"}
Jul 29 19:37:03.924434 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:03.924427    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4" podUID="94b19775-6e4c-48d8-8f3f-5581ea73fc21" containerName="registry-server" containerID="cri-o://7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219" gracePeriod=2
Jul 29 19:37:03.926046 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:03.925984    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-54w2z" event={"ID":"7aeac165-0eb7-4d3a-b6c6-a5e820016ae3","Type":"ContainerStarted","Data":"ace53d12a53cb4d6489d56b6639b20a963989ea20fdb966c1a2b08327520ff58"}
Jul 29 19:37:03.941067 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:03.941024    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4" podStartSLOduration=1.9931725139999998 podStartE2EDuration="3.941007782s" totalImagesPullingTime="1.947835268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:03.940288902 +0000 UTC m=+445.818143232" watchObservedRunningTime="2026-07-29 19:37:03.941007782 +0000 UTC m=+445.818862101"
Jul 29 19:37:03.957176 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:03.957137    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-54w2z" podStartSLOduration=1.565036976 podStartE2EDuration="2.957121747s" totalImagesPullingTime="1.392084771s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:03.955602101 +0000 UTC m=+445.833456430" watchObservedRunningTime="2026-07-29 19:37:03.957121747 +0000 UTC m=+445.834976075"
Jul 29 19:37:04.165596 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.165572    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:04.308435 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.308348    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56kjq\" (UniqueName: \"kubernetes.io/projected/94b19775-6e4c-48d8-8f3f-5581ea73fc21-kube-api-access-56kjq\") pod \"94b19775-6e4c-48d8-8f3f-5581ea73fc21\" (UID: \"94b19775-6e4c-48d8-8f3f-5581ea73fc21\") "
Jul 29 19:37:04.310646 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.310622    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94b19775-6e4c-48d8-8f3f-5581ea73fc21-kube-api-access-56kjq" (OuterVolumeSpecName: "kube-api-access-56kjq") pod "94b19775-6e4c-48d8-8f3f-5581ea73fc21" (UID: "94b19775-6e4c-48d8-8f3f-5581ea73fc21"). InnerVolumeSpecName "kube-api-access-56kjq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:37:04.409305 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.409260    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56kjq\" (UniqueName: \"kubernetes.io/projected/94b19775-6e4c-48d8-8f3f-5581ea73fc21-kube-api-access-56kjq\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:04.935615 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.935583    2580 generic.go:356] "Generic (PLEG): container finished" podID="94b19775-6e4c-48d8-8f3f-5581ea73fc21" containerID="7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219" exitCode=0
Jul 29 19:37:04.936039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.935628    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4"
Jul 29 19:37:04.936039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.935670    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4" event={"ID":"94b19775-6e4c-48d8-8f3f-5581ea73fc21","Type":"ContainerDied","Data":"7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219"}
Jul 29 19:37:04.936039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.935723    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-sdkt4" event={"ID":"94b19775-6e4c-48d8-8f3f-5581ea73fc21","Type":"ContainerDied","Data":"522224f32ff04e81c69b490e96319e51ba0e73c7d17e0b8f5b9e4a69c5814535"}
Jul 29 19:37:04.936039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.935746    2580 scope.go:117] "RemoveContainer" containerID="7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219"
Jul 29 19:37:04.944827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.944798    2580 scope.go:117] "RemoveContainer" containerID="7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219"
Jul 29 19:37:04.945175 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:04.945153    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219\": container with ID starting with 7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219 not found: ID does not exist" containerID="7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219"
Jul 29 19:37:04.945269 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.945182    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219"} err="failed to get container status \"7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219\": rpc error: code = NotFound desc = could not find container \"7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219\": container with ID starting with 7e54626635661ff6c6a4e691ff6b182000f0bf6648538cd2fa1cb039f369d219 not found: ID does not exist"
Jul 29 19:37:04.954564 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.954541    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sdkt4"]
Jul 29 19:37:04.958915 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:04.958894    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-sdkt4"]
Jul 29 19:37:06.716335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:06.716300    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94b19775-6e4c-48d8-8f3f-5581ea73fc21" path="/var/lib/kubelet/pods/94b19775-6e4c-48d8-8f3f-5581ea73fc21/volumes"
Jul 29 19:37:11.798536 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:11.798494    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:11.799066 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:11.798578    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:11.820800 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:11.820772    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:11.991524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:11.991493    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-54w2z"
Jul 29 19:37:20.969268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:20.969232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:30.408281 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:30.408238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:31.231971 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:31.231936    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:31.335858 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:31.335818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:33.060930 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.060896    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"]
Jul 29 19:37:33.061321 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.061277    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94b19775-6e4c-48d8-8f3f-5581ea73fc21" containerName="registry-server"
Jul 29 19:37:33.061321 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.061289    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="94b19775-6e4c-48d8-8f3f-5581ea73fc21" containerName="registry-server"
Jul 29 19:37:33.061392 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.061362    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="94b19775-6e4c-48d8-8f3f-5581ea73fc21" containerName="registry-server"
Jul 29 19:37:33.064405 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.064383    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:33.066249 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.066229    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-vm96j\""
Jul 29 19:37:33.076038 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.075982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"]
Jul 29 19:37:33.174254 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.174220    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/1140c077-9956-4724-bd70-07449f97fb31-kube-api-access-tb9zp\") pod \"limitador-operator-controller-manager-859d7b78bd-z7lll\" (UID: \"1140c077-9956-4724-bd70-07449f97fb31\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:33.239639 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.239604    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:33.274818 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.274785    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/1140c077-9956-4724-bd70-07449f97fb31-kube-api-access-tb9zp\") pod \"limitador-operator-controller-manager-859d7b78bd-z7lll\" (UID: \"1140c077-9956-4724-bd70-07449f97fb31\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:33.288885 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.288858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/1140c077-9956-4724-bd70-07449f97fb31-kube-api-access-tb9zp\") pod \"limitador-operator-controller-manager-859d7b78bd-z7lll\" (UID: \"1140c077-9956-4724-bd70-07449f97fb31\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:33.354945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.354861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:33.375865 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.375823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:33.517514 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:33.517486    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"]
Jul 29 19:37:33.520127 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:33.520097    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1140c077_9956_4724_bd70_07449f97fb31.slice/crio-02e1b362370af74d6686bba02de928196e57f072feedf70e9772bade1f6839b9 WatchSource:0}: Error finding container 02e1b362370af74d6686bba02de928196e57f072feedf70e9772bade1f6839b9: Status 404 returned error can't find the container with id 02e1b362370af74d6686bba02de928196e57f072feedf70e9772bade1f6839b9
Jul 29 19:37:34.016954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:34.016921    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:34.079023 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:34.078965    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll" event={"ID":"1140c077-9956-4724-bd70-07449f97fb31","Type":"ContainerStarted","Data":"02e1b362370af74d6686bba02de928196e57f072feedf70e9772bade1f6839b9"}
Jul 29 19:37:34.118261 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:34.118232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:36.097522 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:36.097488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll" event={"ID":"1140c077-9956-4724-bd70-07449f97fb31","Type":"ContainerStarted","Data":"3e0428f2a209440ee159a27a2a2ba5d3eb84542b9d15045d23279e09b05941ba"}
Jul 29 19:37:36.097916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:36.097623    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:36.120957 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:36.120902    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll" podStartSLOduration=0.647999815 podStartE2EDuration="3.120888484s" totalImagesPullingTime="2.472888669s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:36.119635931 +0000 UTC m=+477.997490260" watchObservedRunningTime="2026-07-29 19:37:36.120888484 +0000 UTC m=+477.998742812"
Jul 29 19:37:38.030683 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.030653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"]
Jul 29 19:37:38.034951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.034932    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.036625 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.036607    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dmlqf\""
Jul 29 19:37:38.048593 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.048570    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"]
Jul 29 19:37:38.117686 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.117644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.117882 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.117763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwnqp\" (UniqueName: \"kubernetes.io/projected/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-kube-api-access-qwnqp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.219225 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.219177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwnqp\" (UniqueName: \"kubernetes.io/projected/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-kube-api-access-qwnqp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.219437 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.219251    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.219641 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.219617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.228106 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.228070    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwnqp\" (UniqueName: \"kubernetes.io/projected/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-kube-api-access-qwnqp\") pod \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.345883 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.345801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:38.420905 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.420857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:38.493773 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:38.493735    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"]
Jul 29 19:37:38.497303 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:38.497274    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9e3cc9b_efc3_4ec2_ae19_40af81f33031.slice/crio-6abb9fa3af28d8167c7c215f825f95c95babb0652fa0c8582749578f9b86e526 WatchSource:0}: Error finding container 6abb9fa3af28d8167c7c215f825f95c95babb0652fa0c8582749578f9b86e526: Status 404 returned error can't find the container with id 6abb9fa3af28d8167c7c215f825f95c95babb0652fa0c8582749578f9b86e526
Jul 29 19:37:39.010185 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:39.010150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:39.113579 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:39.113543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb" event={"ID":"d9e3cc9b-efc3-4ec2-ae19-40af81f33031","Type":"ContainerStarted","Data":"6abb9fa3af28d8167c7c215f825f95c95babb0652fa0c8582749578f9b86e526"}
Jul 29 19:37:40.669474 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:40.669439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:41.049550 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.049469    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"]
Jul 29 19:37:41.053287 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.053271    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:41.055638 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.055617    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-s69qj\""
Jul 29 19:37:41.063482 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.063460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"]
Jul 29 19:37:41.152903 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.152866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzd55\" (UniqueName: \"kubernetes.io/projected/3cec8f23-6e0e-4886-bc86-1c549fabd341-kube-api-access-gzd55\") pod \"authorino-operator-75cc8d5cb7-8mzbx\" (UID: \"3cec8f23-6e0e-4886-bc86-1c549fabd341\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:41.223749 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.223699    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c6967c554-ntwd5"]
Jul 29 19:37:41.227868 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.227839    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.241894 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.241859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6967c554-ntwd5"]
Jul 29 19:37:41.254379 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.254344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzd55\" (UniqueName: \"kubernetes.io/projected/3cec8f23-6e0e-4886-bc86-1c549fabd341-kube-api-access-gzd55\") pod \"authorino-operator-75cc8d5cb7-8mzbx\" (UID: \"3cec8f23-6e0e-4886-bc86-1c549fabd341\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:41.267417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.267389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzd55\" (UniqueName: \"kubernetes.io/projected/3cec8f23-6e0e-4886-bc86-1c549fabd341-kube-api-access-gzd55\") pod \"authorino-operator-75cc8d5cb7-8mzbx\" (UID: \"3cec8f23-6e0e-4886-bc86-1c549fabd341\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:41.342129 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.342044    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:41.355183 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-service-ca\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.355183 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355188    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-oauth-serving-cert\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.355426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xl9h\" (UniqueName: \"kubernetes.io/projected/1672f85e-896b-4c77-b824-710a1934998c-kube-api-access-5xl9h\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.355426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1672f85e-896b-4c77-b824-710a1934998c-console-serving-cert\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.355426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-console-config\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.355426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355371    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-trusted-ca-bundle\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.355583 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.355474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1672f85e-896b-4c77-b824-710a1934998c-console-oauth-config\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.363966 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.363937    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:41.457088 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1672f85e-896b-4c77-b824-710a1934998c-console-serving-cert\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.457263 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-console-config\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.457263 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-trusted-ca-bundle\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.457263 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457160    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1672f85e-896b-4c77-b824-710a1934998c-console-oauth-config\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.457263 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-service-ca\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.457263 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-oauth-serving-cert\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.457527 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.457335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xl9h\" (UniqueName: \"kubernetes.io/projected/1672f85e-896b-4c77-b824-710a1934998c-kube-api-access-5xl9h\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.458122 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.458095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-console-config\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.458246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.458228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-oauth-serving-cert\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.459081 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.459049    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-trusted-ca-bundle\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.459193 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.459143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1672f85e-896b-4c77-b824-710a1934998c-service-ca\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.460947 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.460926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1672f85e-896b-4c77-b824-710a1934998c-console-serving-cert\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.461066 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.461043    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1672f85e-896b-4c77-b824-710a1934998c-console-oauth-config\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.467052 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.466881    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:41.470424 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.470358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xl9h\" (UniqueName: \"kubernetes.io/projected/1672f85e-896b-4c77-b824-710a1934998c-kube-api-access-5xl9h\") pod \"console-c6967c554-ntwd5\" (UID: \"1672f85e-896b-4c77-b824-710a1934998c\") " pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.528931 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.528903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"]
Jul 29 19:37:41.530688 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:41.530655    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cec8f23_6e0e_4886_bc86_1c549fabd341.slice/crio-a3e3a8ff5d9e04b76caf46f1dc8aee60a0f33713deede4b8bfb361bfcbb40514 WatchSource:0}: Error finding container a3e3a8ff5d9e04b76caf46f1dc8aee60a0f33713deede4b8bfb361bfcbb40514: Status 404 returned error can't find the container with id a3e3a8ff5d9e04b76caf46f1dc8aee60a0f33713deede4b8bfb361bfcbb40514
Jul 29 19:37:41.539360 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.539338    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:41.691651 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:41.691626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c6967c554-ntwd5"]
Jul 29 19:37:41.694074 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:41.694046    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1672f85e_896b_4c77_b824_710a1934998c.slice/crio-55b49ab6d06122735a9b7f52cf1a4de4ebfdfc04cc4eb5a315969082c84ccf6a WatchSource:0}: Error finding container 55b49ab6d06122735a9b7f52cf1a4de4ebfdfc04cc4eb5a315969082c84ccf6a: Status 404 returned error can't find the container with id 55b49ab6d06122735a9b7f52cf1a4de4ebfdfc04cc4eb5a315969082c84ccf6a
Jul 29 19:37:42.020934 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.020903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:42.122611 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.122571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:42.131665 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.131639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx" event={"ID":"3cec8f23-6e0e-4886-bc86-1c549fabd341","Type":"ContainerStarted","Data":"a3e3a8ff5d9e04b76caf46f1dc8aee60a0f33713deede4b8bfb361bfcbb40514"}
Jul 29 19:37:42.133300 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.133271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6967c554-ntwd5" event={"ID":"1672f85e-896b-4c77-b824-710a1934998c","Type":"ContainerStarted","Data":"e28501384bfa76cc2addbe6898187c117da1b09f85676a6c80d831782042a55d"}
Jul 29 19:37:42.133450 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.133318    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c6967c554-ntwd5" event={"ID":"1672f85e-896b-4c77-b824-710a1934998c","Type":"ContainerStarted","Data":"55b49ab6d06122735a9b7f52cf1a4de4ebfdfc04cc4eb5a315969082c84ccf6a"}
Jul 29 19:37:42.151181 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.151135    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c6967c554-ntwd5" podStartSLOduration=1.1511189800000001 podStartE2EDuration="1.15111898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:42.149600084 +0000 UTC m=+484.027454411" watchObservedRunningTime="2026-07-29 19:37:42.15111898 +0000 UTC m=+484.028973308"
Jul 29 19:37:42.244928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:42.244889    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:43.067283 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:43.067248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:43.165858 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:43.165818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:44.417027 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.416972    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"]
Jul 29 19:37:44.421211 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.421184    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:44.422948 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.422925    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 29 19:37:44.423164 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.422969    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-drkxx\""
Jul 29 19:37:44.431088 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.430878    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"]
Jul 29 19:37:44.585140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.585097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpwmf\" (UniqueName: \"kubernetes.io/projected/4eb9318a-4701-4713-9eb4-273d5c9c5318-kube-api-access-kpwmf\") pod \"dns-operator-controller-manager-864dd78c4c-w5b9f\" (UID: \"4eb9318a-4701-4713-9eb4-273d5c9c5318\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:44.686128 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.686033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpwmf\" (UniqueName: \"kubernetes.io/projected/4eb9318a-4701-4713-9eb4-273d5c9c5318-kube-api-access-kpwmf\") pod \"dns-operator-controller-manager-864dd78c4c-w5b9f\" (UID: \"4eb9318a-4701-4713-9eb4-273d5c9c5318\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:44.699428 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.699393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpwmf\" (UniqueName: \"kubernetes.io/projected/4eb9318a-4701-4713-9eb4-273d5c9c5318-kube-api-access-kpwmf\") pod \"dns-operator-controller-manager-864dd78c4c-w5b9f\" (UID: \"4eb9318a-4701-4713-9eb4-273d5c9c5318\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:44.736171 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:44.736139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:45.058065 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:45.057946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:45.430878 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:45.430843    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"]
Jul 29 19:37:45.434373 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:45.434344    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eb9318a_4701_4713_9eb4_273d5c9c5318.slice/crio-096ae86db9a95a43ebe0c0e5f91be41b5406ad3d334437b7455229b17a0e02d9 WatchSource:0}: Error finding container 096ae86db9a95a43ebe0c0e5f91be41b5406ad3d334437b7455229b17a0e02d9: Status 404 returned error can't find the container with id 096ae86db9a95a43ebe0c0e5f91be41b5406ad3d334437b7455229b17a0e02d9
Jul 29 19:37:46.161117 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.161070    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb" event={"ID":"d9e3cc9b-efc3-4ec2-ae19-40af81f33031","Type":"ContainerStarted","Data":"3a032fbc939d38c49d22d561a6a4c42bcc6552f98dcc1896a8cfd5dbf8012618"}
Jul 29 19:37:46.161298 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.161136    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:46.162650 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.162625    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f" event={"ID":"4eb9318a-4701-4713-9eb4-273d5c9c5318","Type":"ContainerStarted","Data":"096ae86db9a95a43ebe0c0e5f91be41b5406ad3d334437b7455229b17a0e02d9"}
Jul 29 19:37:46.164640 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.164588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx" event={"ID":"3cec8f23-6e0e-4886-bc86-1c549fabd341","Type":"ContainerStarted","Data":"b422187ef5b87db49aadd0f43866e648b4a9f267f7877a56b42e09f9d6469dd0"}
Jul 29 19:37:46.165078 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.164716    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:46.184103 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.184059    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb" podStartSLOduration=1.403880368 podStartE2EDuration="8.184042585s" totalImagesPullingTime="6.780162217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:46.181609712 +0000 UTC m=+488.059464064" watchObservedRunningTime="2026-07-29 19:37:46.184042585 +0000 UTC m=+488.061896913"
Jul 29 19:37:46.199468 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.199420    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx" podStartSLOduration=1.465817028 podStartE2EDuration="5.199376129s" totalImagesPullingTime="3.733559101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:46.197983732 +0000 UTC m=+488.075838064" watchObservedRunningTime="2026-07-29 19:37:46.199376129 +0000 UTC m=+488.077230460"
Jul 29 19:37:46.420240 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.420153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:46.528097 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:46.528064    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:47.105914 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:47.105879    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:48.184973 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:48.184928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f" event={"ID":"4eb9318a-4701-4713-9eb4-273d5c9c5318","Type":"ContainerStarted","Data":"46d51b01331677d5f4e68fbbd94e3778fe799bbab440295e27eb05426bee3cc1"}
Jul 29 19:37:48.185475 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:48.185069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:48.205583 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:48.205531    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f" podStartSLOduration=2.322500997 podStartE2EDuration="4.205514957s" totalImagesPullingTime="1.88301396s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:37:48.203873087 +0000 UTC m=+490.081727428" watchObservedRunningTime="2026-07-29 19:37:48.205514957 +0000 UTC m=+490.083369284"
Jul 29 19:37:48.352901 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:48.352864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:48.673925 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:48.673895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:49.818584 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:49.818551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:49.886574 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:49.886537    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:51.539714 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:51.539678    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:51.540187 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:51.539741    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:51.544494 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:51.544471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:52.210143 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:52.210115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c6967c554-ntwd5"
Jul 29 19:37:52.273664 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:52.270337    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ffc6c8b4-t7wwh"]
Jul 29 19:37:52.379768 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:52.379735    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:52.480245 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:52.480148    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:57.181111 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:57.181076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-8mzbx"
Jul 29 19:37:57.181570 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:57.181165    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:58.501283 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:58.501246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:58.597945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:58.593339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:37:58.912420 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:58.912378    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"]
Jul 29 19:37:58.916225 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:58.916205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:58.940523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:58.940489    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"]
Jul 29 19:37:59.020065 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.020020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75bj2\" (UniqueName: \"kubernetes.io/projected/f03dc169-8fbf-4172-af32-17acf2134d53-kube-api-access-75bj2\") pod \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" (UID: \"f03dc169-8fbf-4172-af32-17acf2134d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.020251 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.020111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f03dc169-8fbf-4172-af32-17acf2134d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" (UID: \"f03dc169-8fbf-4172-af32-17acf2134d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.062478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.062438    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"]
Jul 29 19:37:59.062748 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.062729    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-75bj2], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9" podUID="f03dc169-8fbf-4172-af32-17acf2134d53"
Jul 29 19:37:59.066562 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.066537    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"]
Jul 29 19:37:59.076823 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.076800    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"]
Jul 29 19:37:59.077025 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.076984    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" containerName="manager" containerID="cri-o://3a032fbc939d38c49d22d561a6a4c42bcc6552f98dcc1896a8cfd5dbf8012618" gracePeriod=2
Jul 29 19:37:59.081292 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.081254    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"]
Jul 29 19:37:59.092361 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.092329    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"]
Jul 29 19:37:59.092943 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.092910    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" containerName="manager"
Jul 29 19:37:59.093158 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.092964    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" containerName="manager"
Jul 29 19:37:59.093158 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.093125    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" containerName="manager"
Jul 29 19:37:59.102671 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.102642    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"]
Jul 29 19:37:59.102779 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.102737    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"]
Jul 29 19:37:59.102842 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.102771    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.103743 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.103357    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll" podUID="1140c077-9956-4724-bd70-07449f97fb31" containerName="manager" containerID="cri-o://3e0428f2a209440ee159a27a2a2ba5d3eb84542b9d15045d23279e09b05941ba" gracePeriod=2
Jul 29 19:37:59.112089 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.112060    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"]
Jul 29 19:37:59.112715 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.112692    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1140c077-9956-4724-bd70-07449f97fb31" containerName="manager"
Jul 29 19:37:59.112796 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.112718    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1140c077-9956-4724-bd70-07449f97fb31" containerName="manager"
Jul 29 19:37:59.112875 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.112862    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1140c077-9956-4724-bd70-07449f97fb31" containerName="manager"
Jul 29 19:37:59.116396 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.116350    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.116527 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.116476    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"]
Jul 29 19:37:59.121403 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.121370    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdc22321-2431-4e2a-a299-24e981caa24d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9cw7g\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.121523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.121440    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f03dc169-8fbf-4172-af32-17acf2134d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" (UID: \"f03dc169-8fbf-4172-af32-17acf2134d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.121523 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.121486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25r7t\" (UniqueName: \"kubernetes.io/projected/bdc22321-2431-4e2a-a299-24e981caa24d-kube-api-access-25r7t\") pod \"kuadrant-operator-controller-manager-55467d9f7-9cw7g\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.121622 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.121591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75bj2\" (UniqueName: \"kubernetes.io/projected/f03dc169-8fbf-4172-af32-17acf2134d53-kube-api-access-75bj2\") pod \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" (UID: \"f03dc169-8fbf-4172-af32-17acf2134d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.121859 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.121834    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f03dc169-8fbf-4172-af32-17acf2134d53-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" (UID: \"f03dc169-8fbf-4172-af32-17acf2134d53\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.122618 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.122595    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"]
Jul 29 19:37:59.126876 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.126855    2580 projected.go:196] Error preparing data for projected volume kube-api-access-75bj2 for pod kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9: failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-tc2t9" not found
Jul 29 19:37:59.126981 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.126919    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"]
Jul 29 19:37:59.126981 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.126934    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f03dc169-8fbf-4172-af32-17acf2134d53-kube-api-access-75bj2 podName:f03dc169-8fbf-4172-af32-17acf2134d53 nodeName:}" failed. No retries permitted until 2026-07-29 19:37:59.626912036 +0000 UTC m=+501.504766346 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-75bj2" (UniqueName: "kubernetes.io/projected/f03dc169-8fbf-4172-af32-17acf2134d53-kube-api-access-75bj2") pod "kuadrant-operator-controller-manager-55467d9f7-tc2t9" (UID: "f03dc169-8fbf-4172-af32-17acf2134d53") : failed to fetch token: pod "kuadrant-operator-controller-manager-55467d9f7-tc2t9" not found
Jul 29 19:37:59.127135 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.127066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:37:59.128478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.128455    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"]
Jul 29 19:37:59.139370 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.139341    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"]
Jul 29 19:37:59.145608 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.145576    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.174741 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.174664    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.176123 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.176095    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.192750 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.192713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-w5b9f"
Jul 29 19:37:59.195124 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.195081    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.207611 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.207585    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-25r7t], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g" podUID="bdc22321-2431-4e2a-a299-24e981caa24d"
Jul 29 19:37:59.216154 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.216119    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.222797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.222245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25r7t\" (UniqueName: \"kubernetes.io/projected/bdc22321-2431-4e2a-a299-24e981caa24d-kube-api-access-25r7t\") pod \"kuadrant-operator-controller-manager-55467d9f7-9cw7g\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.222797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.222316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b61340c-44b2-4e12-acc2-b0822f98ca4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k47z\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.222797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.222348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dndjk\" (UniqueName: \"kubernetes.io/projected/4b61340c-44b2-4e12-acc2-b0822f98ca4e-kube-api-access-dndjk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k47z\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.222797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.222383    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5b82\" (UniqueName: \"kubernetes.io/projected/53c58309-bd86-4cfb-9b0d-db48f15fbae5-kube-api-access-h5b82\") pod \"limitador-operator-controller-manager-859d7b78bd-8hlt2\" (UID: \"53c58309-bd86-4cfb-9b0d-db48f15fbae5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:37:59.222797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.222491    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdc22321-2431-4e2a-a299-24e981caa24d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9cw7g\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.223473 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.222862    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdc22321-2431-4e2a-a299-24e981caa24d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-9cw7g\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.236165 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.236124    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25r7t\" (UniqueName: \"kubernetes.io/projected/bdc22321-2431-4e2a-a299-24e981caa24d-kube-api-access-25r7t\") pod \"kuadrant-operator-controller-manager-55467d9f7-9cw7g\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.249036 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.248950    2580 generic.go:356] "Generic (PLEG): container finished" podID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" containerID="3a032fbc939d38c49d22d561a6a4c42bcc6552f98dcc1896a8cfd5dbf8012618" exitCode=0
Jul 29 19:37:59.251075 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.251046    2580 generic.go:356] "Generic (PLEG): container finished" podID="1140c077-9956-4724-bd70-07449f97fb31" containerID="3e0428f2a209440ee159a27a2a2ba5d3eb84542b9d15045d23279e09b05941ba" exitCode=0
Jul 29 19:37:59.251207 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.251194    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.251286 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.251195    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.253723 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.253693    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.255203 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.255172    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.256435 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.256404    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.260552 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.260534    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:37:59.265972 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.265931    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.281941 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.281908    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.283716 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.283682    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.285551 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.285523    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.323378 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323347    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdc22321-2431-4e2a-a299-24e981caa24d-extensions-socket-volume\") pod \"bdc22321-2431-4e2a-a299-24e981caa24d\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") "
Jul 29 19:37:59.323538 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f03dc169-8fbf-4172-af32-17acf2134d53-extensions-socket-volume\") pod \"f03dc169-8fbf-4172-af32-17acf2134d53\" (UID: \"f03dc169-8fbf-4172-af32-17acf2134d53\") "
Jul 29 19:37:59.323538 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323486    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25r7t\" (UniqueName: \"kubernetes.io/projected/bdc22321-2431-4e2a-a299-24e981caa24d-kube-api-access-25r7t\") pod \"bdc22321-2431-4e2a-a299-24e981caa24d\" (UID: \"bdc22321-2431-4e2a-a299-24e981caa24d\") "
Jul 29 19:37:59.323632 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdc22321-2431-4e2a-a299-24e981caa24d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bdc22321-2431-4e2a-a299-24e981caa24d" (UID: "bdc22321-2431-4e2a-a299-24e981caa24d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:37:59.323712 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323686    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b61340c-44b2-4e12-acc2-b0822f98ca4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k47z\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.323765 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dndjk\" (UniqueName: \"kubernetes.io/projected/4b61340c-44b2-4e12-acc2-b0822f98ca4e-kube-api-access-dndjk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k47z\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.323765 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f03dc169-8fbf-4172-af32-17acf2134d53-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f03dc169-8fbf-4172-af32-17acf2134d53" (UID: "f03dc169-8fbf-4172-af32-17acf2134d53"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:37:59.323765 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5b82\" (UniqueName: \"kubernetes.io/projected/53c58309-bd86-4cfb-9b0d-db48f15fbae5-kube-api-access-h5b82\") pod \"limitador-operator-controller-manager-859d7b78bd-8hlt2\" (UID: \"53c58309-bd86-4cfb-9b0d-db48f15fbae5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:37:59.323932 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323830    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f03dc169-8fbf-4172-af32-17acf2134d53-extensions-socket-volume\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.323932 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323841    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bdc22321-2431-4e2a-a299-24e981caa24d-extensions-socket-volume\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.323932 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.323851    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-75bj2\" (UniqueName: \"kubernetes.io/projected/f03dc169-8fbf-4172-af32-17acf2134d53-kube-api-access-75bj2\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.324141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.324122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b61340c-44b2-4e12-acc2-b0822f98ca4e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k47z\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.327145 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.327122    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdc22321-2431-4e2a-a299-24e981caa24d-kube-api-access-25r7t" (OuterVolumeSpecName: "kube-api-access-25r7t") pod "bdc22321-2431-4e2a-a299-24e981caa24d" (UID: "bdc22321-2431-4e2a-a299-24e981caa24d"). InnerVolumeSpecName "kube-api-access-25r7t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:37:59.335697 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.335674    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5b82\" (UniqueName: \"kubernetes.io/projected/53c58309-bd86-4cfb-9b0d-db48f15fbae5-kube-api-access-h5b82\") pod \"limitador-operator-controller-manager-859d7b78bd-8hlt2\" (UID: \"53c58309-bd86-4cfb-9b0d-db48f15fbae5\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:37:59.337093 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.337069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dndjk\" (UniqueName: \"kubernetes.io/projected/4b61340c-44b2-4e12-acc2-b0822f98ca4e-kube-api-access-dndjk\") pod \"kuadrant-operator-controller-manager-8684c9ff49-7k47z\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.353619 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.353599    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.355394 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.355365    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.356677 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.356657    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.357152 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.357136    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.357931 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.357912    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.359158 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.359139    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:37:59.360419 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.360401    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:37:59.361835 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:37:59.361812    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:37:59.425203 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.425110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/1140c077-9956-4724-bd70-07449f97fb31-kube-api-access-tb9zp\") pod \"1140c077-9956-4724-bd70-07449f97fb31\" (UID: \"1140c077-9956-4724-bd70-07449f97fb31\") "
Jul 29 19:37:59.425203 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.425164    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwnqp\" (UniqueName: \"kubernetes.io/projected/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-kube-api-access-qwnqp\") pod \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") "
Jul 29 19:37:59.425431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.425225    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-extensions-socket-volume\") pod \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\" (UID: \"d9e3cc9b-efc3-4ec2-ae19-40af81f33031\") "
Jul 29 19:37:59.425431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.425399    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25r7t\" (UniqueName: \"kubernetes.io/projected/bdc22321-2431-4e2a-a299-24e981caa24d-kube-api-access-25r7t\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.425748 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.425725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d9e3cc9b-efc3-4ec2-ae19-40af81f33031" (UID: "d9e3cc9b-efc3-4ec2-ae19-40af81f33031"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:37:59.427355 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.427331    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1140c077-9956-4724-bd70-07449f97fb31-kube-api-access-tb9zp" (OuterVolumeSpecName: "kube-api-access-tb9zp") pod "1140c077-9956-4724-bd70-07449f97fb31" (UID: "1140c077-9956-4724-bd70-07449f97fb31"). InnerVolumeSpecName "kube-api-access-tb9zp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:37:59.427432 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.427374    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-kube-api-access-qwnqp" (OuterVolumeSpecName: "kube-api-access-qwnqp") pod "d9e3cc9b-efc3-4ec2-ae19-40af81f33031" (UID: "d9e3cc9b-efc3-4ec2-ae19-40af81f33031"). InnerVolumeSpecName "kube-api-access-qwnqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:37:59.518098 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.518056    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:37:59.524721 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.524696    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:37:59.526197 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.526173    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb9zp\" (UniqueName: \"kubernetes.io/projected/1140c077-9956-4724-bd70-07449f97fb31-kube-api-access-tb9zp\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.526288 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.526199    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwnqp\" (UniqueName: \"kubernetes.io/projected/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-kube-api-access-qwnqp\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.526288 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.526209    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d9e3cc9b-efc3-4ec2-ae19-40af81f33031-extensions-socket-volume\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:37:59.672730 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.672695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"]
Jul 29 19:37:59.676254 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:59.676176    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b61340c_44b2_4e12_acc2_b0822f98ca4e.slice/crio-b90cdf5885e86c84cc6029a19f6d29a3656f92ec779fa77a56098343124636d7 WatchSource:0}: Error finding container b90cdf5885e86c84cc6029a19f6d29a3656f92ec779fa77a56098343124636d7: Status 404 returned error can't find the container with id b90cdf5885e86c84cc6029a19f6d29a3656f92ec779fa77a56098343124636d7
Jul 29 19:37:59.702199 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:37:59.702176    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"]
Jul 29 19:37:59.704635 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:37:59.704609    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53c58309_bd86_4cfb_9b0d_db48f15fbae5.slice/crio-0d8c8eaaf694bcb5ae0e6f3e59611dcedbd863499cc9a469fe0a1eb576d67b87 WatchSource:0}: Error finding container 0d8c8eaaf694bcb5ae0e6f3e59611dcedbd863499cc9a469fe0a1eb576d67b87: Status 404 returned error can't find the container with id 0d8c8eaaf694bcb5ae0e6f3e59611dcedbd863499cc9a469fe0a1eb576d67b87
Jul 29 19:38:00.257264 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.257234    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:38:00.257264 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.257238    2580 scope.go:117] "RemoveContainer" containerID="3a032fbc939d38c49d22d561a6a4c42bcc6552f98dcc1896a8cfd5dbf8012618"
Jul 29 19:38:00.259482 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.259453    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z" event={"ID":"4b61340c-44b2-4e12-acc2-b0822f98ca4e","Type":"ContainerStarted","Data":"380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4"}
Jul 29 19:38:00.259602 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.259543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z" event={"ID":"4b61340c-44b2-4e12-acc2-b0822f98ca4e","Type":"ContainerStarted","Data":"b90cdf5885e86c84cc6029a19f6d29a3656f92ec779fa77a56098343124636d7"}
Jul 29 19:38:00.259602 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.259573    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:38:00.259815 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.259783    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:38:00.261051 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.261023    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:38:00.261636 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.261603    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:38:00.263247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.262810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2" event={"ID":"53c58309-bd86-4cfb-9b0d-db48f15fbae5","Type":"ContainerStarted","Data":"7d49858e181a7a5f831aaad96da29c452c2b790cf569c886984ec43901ae7596"}
Jul 29 19:38:00.263247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.262830    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:38:00.263247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.262848    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"
Jul 29 19:38:00.263247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.262864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2" event={"ID":"53c58309-bd86-4cfb-9b0d-db48f15fbae5","Type":"ContainerStarted","Data":"0d8c8eaaf694bcb5ae0e6f3e59611dcedbd863499cc9a469fe0a1eb576d67b87"}
Jul 29 19:38:00.263247 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.263037    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:38:00.263671 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.263645    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:38:00.265317 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.265288    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:38:00.269820 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.269801    2580 scope.go:117] "RemoveContainer" containerID="3e0428f2a209440ee159a27a2a2ba5d3eb84542b9d15045d23279e09b05941ba"
Jul 29 19:38:00.305654 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.305597    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2" podStartSLOduration=1.30558108 podStartE2EDuration="1.30558108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:00.298768334 +0000 UTC m=+502.176622663" watchObservedRunningTime="2026-07-29 19:38:00.30558108 +0000 UTC m=+502.183435408"
Jul 29 19:38:00.330738 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.330695    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:38:00.331478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.331436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z" podStartSLOduration=1.331422455 podStartE2EDuration="1.331422455s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:37:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:00.329107466 +0000 UTC m=+502.206961794" watchObservedRunningTime="2026-07-29 19:38:00.331422455 +0000 UTC m=+502.209276782"
Jul 29 19:38:00.331945 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.331920    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:38:00.337118 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.337092    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-l84cb\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-l84cb"
Jul 29 19:38:00.338583 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.338557    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-z7lll\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="1140c077-9956-4724-bd70-07449f97fb31" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-z7lll"
Jul 29 19:38:00.339961 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:00.339938    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-tc2t9\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-tc2t9"
Jul 29 19:38:00.387285 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.387254    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"]
Jul 29 19:38:00.396524 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.396493    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-9cw7g"]
Jul 29 19:38:00.716442 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.716353    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1140c077-9956-4724-bd70-07449f97fb31" path="/var/lib/kubelet/pods/1140c077-9956-4724-bd70-07449f97fb31/volumes"
Jul 29 19:38:00.716927 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.716903    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdc22321-2431-4e2a-a299-24e981caa24d" path="/var/lib/kubelet/pods/bdc22321-2431-4e2a-a299-24e981caa24d/volumes"
Jul 29 19:38:00.717220 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.717203    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9e3cc9b-efc3-4ec2-ae19-40af81f33031" path="/var/lib/kubelet/pods/d9e3cc9b-efc3-4ec2-ae19-40af81f33031/volumes"
Jul 29 19:38:00.717551 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.717535    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f03dc169-8fbf-4172-af32-17acf2134d53" path="/var/lib/kubelet/pods/f03dc169-8fbf-4172-af32-17acf2134d53/volumes"
Jul 29 19:38:00.932666 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:00.932637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:01.036643 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:01.036553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:05.234926 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:05.234888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:05.337241 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:05.337208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:06.345150 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:06.345113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:06.445385 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:06.445353    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:07.445713 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:07.445677    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:07.500329 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:07.500285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:11.272509 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:11.272467    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-8hlt2"
Jul 29 19:38:11.273013 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:11.272546    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:38:16.267705 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:16.267667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:16.373613 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:16.373582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:17.297254 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.297219    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68ffc6c8b4-t7wwh" podUID="ef6d24bd-856b-45ab-9a3d-a496a1983fdc" containerName="console" containerID="cri-o://23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a" gracePeriod=15
Jul 29 19:38:17.549118 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.549092    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:38:17.695891 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.695810    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-config\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.695891 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.695854    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-trusted-ca-bundle\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.696172 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.695911    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-oauth-serving-cert\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.696172 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.695937    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-serving-cert\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.696172 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.695955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-oauth-config\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.696172 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.696021    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-service-ca\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.696172 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.696052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h2g6\" (UniqueName: \"kubernetes.io/projected/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-kube-api-access-4h2g6\") pod \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\" (UID: \"ef6d24bd-856b-45ab-9a3d-a496a1983fdc\") "
Jul 29 19:38:17.696431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.696294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-config" (OuterVolumeSpecName: "console-config") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:38:17.696431 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.696379    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:38:17.696533 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.696435    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:38:17.696533 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.696497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-service-ca" (OuterVolumeSpecName: "service-ca") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:38:17.698368 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.698339    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-kube-api-access-4h2g6" (OuterVolumeSpecName: "kube-api-access-4h2g6") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "kube-api-access-4h2g6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:38:17.698368 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.698359    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:38:17.698500 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.698410    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ef6d24bd-856b-45ab-9a3d-a496a1983fdc" (UID: "ef6d24bd-856b-45ab-9a3d-a496a1983fdc"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:38:17.797600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797555    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-trusted-ca-bundle\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:17.797600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797594    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-oauth-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:17.797600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797608    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-serving-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:17.797600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797617    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-oauth-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:17.797871 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797626    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:17.797871 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797634    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h2g6\" (UniqueName: \"kubernetes.io/projected/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-kube-api-access-4h2g6\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:17.797871 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:17.797642    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ef6d24bd-856b-45ab-9a3d-a496a1983fdc-console-config\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:18.360540 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.360503    2580 generic.go:356] "Generic (PLEG): container finished" podID="ef6d24bd-856b-45ab-9a3d-a496a1983fdc" containerID="23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a" exitCode=0
Jul 29 19:38:18.361046 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.360558    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68ffc6c8b4-t7wwh"
Jul 29 19:38:18.361046 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.360587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc6c8b4-t7wwh" event={"ID":"ef6d24bd-856b-45ab-9a3d-a496a1983fdc","Type":"ContainerDied","Data":"23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a"}
Jul 29 19:38:18.361046 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.360640    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68ffc6c8b4-t7wwh" event={"ID":"ef6d24bd-856b-45ab-9a3d-a496a1983fdc","Type":"ContainerDied","Data":"99900f41791dc07bfaec975c3c42e0395857f51fe1a16b23203e1bfca8190542"}
Jul 29 19:38:18.361046 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.360660    2580 scope.go:117] "RemoveContainer" containerID="23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a"
Jul 29 19:38:18.369911 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.369893    2580 scope.go:117] "RemoveContainer" containerID="23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a"
Jul 29 19:38:18.370221 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:18.370184    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a\": container with ID starting with 23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a not found: ID does not exist" containerID="23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a"
Jul 29 19:38:18.370302 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.370232    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a"} err="failed to get container status \"23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a\": rpc error: code = NotFound desc = could not find container \"23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a\": container with ID starting with 23002e072911486267cc3d2124c45c8f07efa0ca02f3092b2fe054aeaa0a7d7a not found: ID does not exist"
Jul 29 19:38:18.382959 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.382933    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68ffc6c8b4-t7wwh"]
Jul 29 19:38:18.391110 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.391089    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68ffc6c8b4-t7wwh"]
Jul 29 19:38:18.716108 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.716068    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef6d24bd-856b-45ab-9a3d-a496a1983fdc" path="/var/lib/kubelet/pods/ef6d24bd-856b-45ab-9a3d-a496a1983fdc/volumes"
Jul 29 19:38:18.728600 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.728562    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:18.831822 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:18.831787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:23.082967 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:23.082931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:23.183464 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:23.183430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:26.676669 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:26.676636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:26.739916 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:26.739886    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:30.923268 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:30.923183    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"]
Jul 29 19:38:30.923667 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:30.923387    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z" podUID="4b61340c-44b2-4e12-acc2-b0822f98ca4e" containerName="manager" containerID="cri-o://380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4" gracePeriod=10
Jul 29 19:38:31.170775 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.170748    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:38:31.321131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.321048    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b61340c-44b2-4e12-acc2-b0822f98ca4e-extensions-socket-volume\") pod \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") "
Jul 29 19:38:31.321279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.321182    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dndjk\" (UniqueName: \"kubernetes.io/projected/4b61340c-44b2-4e12-acc2-b0822f98ca4e-kube-api-access-dndjk\") pod \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\" (UID: \"4b61340c-44b2-4e12-acc2-b0822f98ca4e\") "
Jul 29 19:38:31.321526 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.321493    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b61340c-44b2-4e12-acc2-b0822f98ca4e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b61340c-44b2-4e12-acc2-b0822f98ca4e" (UID: "4b61340c-44b2-4e12-acc2-b0822f98ca4e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 19:38:31.323327 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.323309    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b61340c-44b2-4e12-acc2-b0822f98ca4e-kube-api-access-dndjk" (OuterVolumeSpecName: "kube-api-access-dndjk") pod "4b61340c-44b2-4e12-acc2-b0822f98ca4e" (UID: "4b61340c-44b2-4e12-acc2-b0822f98ca4e"). InnerVolumeSpecName "kube-api-access-dndjk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:38:31.421877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.421834    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dndjk\" (UniqueName: \"kubernetes.io/projected/4b61340c-44b2-4e12-acc2-b0822f98ca4e-kube-api-access-dndjk\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:31.421877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.421874    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b61340c-44b2-4e12-acc2-b0822f98ca4e-extensions-socket-volume\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:31.432694 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.432664    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b61340c-44b2-4e12-acc2-b0822f98ca4e" containerID="380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4" exitCode=0
Jul 29 19:38:31.432822 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.432713    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"
Jul 29 19:38:31.432822 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.432753    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z" event={"ID":"4b61340c-44b2-4e12-acc2-b0822f98ca4e","Type":"ContainerDied","Data":"380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4"}
Jul 29 19:38:31.432822 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.432805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z" event={"ID":"4b61340c-44b2-4e12-acc2-b0822f98ca4e","Type":"ContainerDied","Data":"b90cdf5885e86c84cc6029a19f6d29a3656f92ec779fa77a56098343124636d7"}
Jul 29 19:38:31.432927 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.432829    2580 scope.go:117] "RemoveContainer" containerID="380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4"
Jul 29 19:38:31.441633 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.441622    2580 scope.go:117] "RemoveContainer" containerID="380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4"
Jul 29 19:38:31.441882 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:31.441866    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4\": container with ID starting with 380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4 not found: ID does not exist" containerID="380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4"
Jul 29 19:38:31.441931 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.441888    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4"} err="failed to get container status \"380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4\": rpc error: code = NotFound desc = could not find container \"380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4\": container with ID starting with 380586c5799057be29959d7c21d1eb20dc1cdce63a64d4f77d2ce11d096f0bf4 not found: ID does not exist"
Jul 29 19:38:31.455422 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.455394    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"]
Jul 29 19:38:31.467674 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:31.467653    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-7k47z"]
Jul 29 19:38:32.715734 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:32.715693    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b61340c-44b2-4e12-acc2-b0822f98ca4e" path="/var/lib/kubelet/pods/4b61340c-44b2-4e12-acc2-b0822f98ca4e/volumes"
Jul 29 19:38:36.621350 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:36.621311    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:36.673186 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:36.673156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:36.862243 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:36.862204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:36.921134 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:36.921104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:40.687016 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:40.686967    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:40.788009 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:40.787953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:45.507819 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:45.507784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:45.609277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:45.609242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:47.259906 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.259874    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"]
Jul 29 19:38:47.260295 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.260282    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b61340c-44b2-4e12-acc2-b0822f98ca4e" containerName="manager"
Jul 29 19:38:47.260295 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.260296    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b61340c-44b2-4e12-acc2-b0822f98ca4e" containerName="manager"
Jul 29 19:38:47.260374 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.260312    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef6d24bd-856b-45ab-9a3d-a496a1983fdc" containerName="console"
Jul 29 19:38:47.260374 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.260317    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef6d24bd-856b-45ab-9a3d-a496a1983fdc" containerName="console"
Jul 29 19:38:47.260437 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.260393    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef6d24bd-856b-45ab-9a3d-a496a1983fdc" containerName="console"
Jul 29 19:38:47.260437 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.260403    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b61340c-44b2-4e12-acc2-b0822f98ca4e" containerName="manager"
Jul 29 19:38:47.263790 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.263772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.265469 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.265449    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 29 19:38:47.265598 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.265521    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xcsgm\""
Jul 29 19:38:47.277280 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.277257    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"]
Jul 29 19:38:47.297522 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.297489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:47.368789 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.368750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.368954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.368801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f016396-42e6-4348-a688-dc65be237fa3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.368954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.368854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xcm2\" (UniqueName: \"kubernetes.io/projected/4f016396-42e6-4348-a688-dc65be237fa3-kube-api-access-4xcm2\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.368954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.368891    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.368954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.368940    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f016396-42e6-4348-a688-dc65be237fa3-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.369141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.368973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.369141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.369017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f016396-42e6-4348-a688-dc65be237fa3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.369141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.369069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.369141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.369100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470159 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470120    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f016396-42e6-4348-a688-dc65be237fa3-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f016396-42e6-4348-a688-dc65be237fa3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f016396-42e6-4348-a688-dc65be237fa3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xcm2\" (UniqueName: \"kubernetes.io/projected/4f016396-42e6-4348-a688-dc65be237fa3-kube-api-access-4xcm2\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470401    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470739    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470836 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470759    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470930 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.470930 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.470874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.471435 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.471409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f016396-42e6-4348-a688-dc65be237fa3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.472677 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.472657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f016396-42e6-4348-a688-dc65be237fa3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.472914 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.472894    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f016396-42e6-4348-a688-dc65be237fa3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.479183 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.479156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f016396-42e6-4348-a688-dc65be237fa3-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.479370 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.479349    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xcm2\" (UniqueName: \"kubernetes.io/projected/4f016396-42e6-4348-a688-dc65be237fa3-kube-api-access-4xcm2\") pod \"maas-default-gateway-openshift-default-5ccb968c66-7rzcx\" (UID: \"4f016396-42e6-4348-a688-dc65be237fa3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.575344 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.575256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:47.645368 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.645337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:47.717574 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:47.717542    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"]
Jul 29 19:38:47.720470 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:38:47.720439    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f016396_42e6_4348_a688_dc65be237fa3.slice/crio-ea14ab7d8e71c6c09c7d13e403c870dde7a9f844e573df915e1a241260e8c25e WatchSource:0}: Error finding container ea14ab7d8e71c6c09c7d13e403c870dde7a9f844e573df915e1a241260e8c25e: Status 404 returned error can't find the container with id ea14ab7d8e71c6c09c7d13e403c870dde7a9f844e573df915e1a241260e8c25e
Jul 29 19:38:48.526864 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:48.526828    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" event={"ID":"4f016396-42e6-4348-a688-dc65be237fa3","Type":"ContainerStarted","Data":"ea14ab7d8e71c6c09c7d13e403c870dde7a9f844e573df915e1a241260e8c25e"}
Jul 29 19:38:50.608942 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:50.608905    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:38:50.609210 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:50.608975    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:38:50.609210 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:50.609018    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:38:51.545797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:51.545757    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" event={"ID":"4f016396-42e6-4348-a688-dc65be237fa3","Type":"ContainerStarted","Data":"8b466fc4ad314236bab58ad4a36720603001d9a549afa48571d5576a6ec9b510"}
Jul 29 19:38:51.568592 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:51.568534    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" podStartSLOduration=1.682519611 podStartE2EDuration="4.568515972s" totalImagesPullingTime="2.885996361s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:38:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:51.566914444 +0000 UTC m=+553.444768774" watchObservedRunningTime="2026-07-29 19:38:51.568515972 +0000 UTC m=+553.446370278"
Jul 29 19:38:51.576188 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:51.576154    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:51.581086 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:51.581065    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:52.551962 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.551922    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:52.552816 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.552798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:38:52.560746 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.560722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:52.567223 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.567204    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:38:52.569937 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.569921    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.571589 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.571570    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 29 19:38:52.571681 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.571622    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-h7xjc\""
Jul 29 19:38:52.580765 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.580742    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:38:52.618558 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.618522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2xv8\" (UniqueName: \"kubernetes.io/projected/6c077cdc-8304-4e46-b45b-aa15fde400bf-kube-api-access-g2xv8\") pod \"limitador-limitador-64c94fd64f-vd8pl\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.618737 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.618720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c077cdc-8304-4e46-b45b-aa15fde400bf-config-file\") pod \"limitador-limitador-64c94fd64f-vd8pl\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.659306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.659264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:38:52.669817 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.669782    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:52.716578 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.716553    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:38:52.718904 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.718883    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:52.719372 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.719355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c077cdc-8304-4e46-b45b-aa15fde400bf-config-file\") pod \"limitador-limitador-64c94fd64f-vd8pl\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.719478 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.719412    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2xv8\" (UniqueName: \"kubernetes.io/projected/6c077cdc-8304-4e46-b45b-aa15fde400bf-kube-api-access-g2xv8\") pod \"limitador-limitador-64c94fd64f-vd8pl\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.719951 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.719929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c077cdc-8304-4e46-b45b-aa15fde400bf-config-file\") pod \"limitador-limitador-64c94fd64f-vd8pl\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.727864 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.727837    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:38:52.730042 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.729987    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2xv8\" (UniqueName: \"kubernetes.io/projected/6c077cdc-8304-4e46-b45b-aa15fde400bf-kube-api-access-g2xv8\") pod \"limitador-limitador-64c94fd64f-vd8pl\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.741308 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.741284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:52.752316 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.752292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:38:52.820712 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.820635    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/56c9f00e-289b-4e20-a364-c4a189c6c469-config-file\") pod \"limitador-limitador-54d6d988d5-dvfvg\" (UID: \"56c9f00e-289b-4e20-a364-c4a189c6c469\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:52.820852 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.820718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxdzg\" (UniqueName: \"kubernetes.io/projected/56c9f00e-289b-4e20-a364-c4a189c6c469-kube-api-access-mxdzg\") pod \"limitador-limitador-54d6d988d5-dvfvg\" (UID: \"56c9f00e-289b-4e20-a364-c4a189c6c469\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:52.880632 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.880602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:52.922066 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.922033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/56c9f00e-289b-4e20-a364-c4a189c6c469-config-file\") pod \"limitador-limitador-54d6d988d5-dvfvg\" (UID: \"56c9f00e-289b-4e20-a364-c4a189c6c469\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:52.922256 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.922133    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxdzg\" (UniqueName: \"kubernetes.io/projected/56c9f00e-289b-4e20-a364-c4a189c6c469-kube-api-access-mxdzg\") pod \"limitador-limitador-54d6d988d5-dvfvg\" (UID: \"56c9f00e-289b-4e20-a364-c4a189c6c469\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:52.922663 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.922643    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/56c9f00e-289b-4e20-a364-c4a189c6c469-config-file\") pod \"limitador-limitador-54d6d988d5-dvfvg\" (UID: \"56c9f00e-289b-4e20-a364-c4a189c6c469\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:52.932479 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:52.932448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxdzg\" (UniqueName: \"kubernetes.io/projected/56c9f00e-289b-4e20-a364-c4a189c6c469-kube-api-access-mxdzg\") pod \"limitador-limitador-54d6d988d5-dvfvg\" (UID: \"56c9f00e-289b-4e20-a364-c4a189c6c469\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:53.022180 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.022086    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:38:53.025119 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:38:53.025086    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c077cdc_8304_4e46_b45b_aa15fde400bf.slice/crio-05fe7e031f566d1424d7071017f8cd3deeb5b97d2ca604279ac2dd4a567fc8e7 WatchSource:0}: Error finding container 05fe7e031f566d1424d7071017f8cd3deeb5b97d2ca604279ac2dd4a567fc8e7: Status 404 returned error can't find the container with id 05fe7e031f566d1424d7071017f8cd3deeb5b97d2ca604279ac2dd4a567fc8e7
Jul 29 19:38:53.031751 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.031729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:53.175424 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.175400    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:38:53.177840 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:38:53.177809    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56c9f00e_289b_4e20_a364_c4a189c6c469.slice/crio-cd9816002d7bb8ba0d68e72a390624106e078bc6c091536bba8d58e8dcee0171 WatchSource:0}: Error finding container cd9816002d7bb8ba0d68e72a390624106e078bc6c091536bba8d58e8dcee0171: Status 404 returned error can't find the container with id cd9816002d7bb8ba0d68e72a390624106e078bc6c091536bba8d58e8dcee0171
Jul 29 19:38:53.488928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.488888    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-q65tg"]
Jul 29 19:38:53.492235 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.492209    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:53.494045 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.494023    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ldhdt\""
Jul 29 19:38:53.503506 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.503485    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q65tg"]
Jul 29 19:38:53.519787 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.519758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:53.558644 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.558609    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg" event={"ID":"56c9f00e-289b-4e20-a364-c4a189c6c469","Type":"ContainerStarted","Data":"cd9816002d7bb8ba0d68e72a390624106e078bc6c091536bba8d58e8dcee0171"}
Jul 29 19:38:53.559748 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.559724    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl" event={"ID":"6c077cdc-8304-4e46-b45b-aa15fde400bf","Type":"ContainerStarted","Data":"05fe7e031f566d1424d7071017f8cd3deeb5b97d2ca604279ac2dd4a567fc8e7"}
Jul 29 19:38:53.582020 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.579027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:38:53.629168 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.629125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrxsc\" (UniqueName: \"kubernetes.io/projected/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5-kube-api-access-jrxsc\") pod \"authorino-58cd955675-q65tg\" (UID: \"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5\") " pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:53.730635 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.730591    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrxsc\" (UniqueName: \"kubernetes.io/projected/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5-kube-api-access-jrxsc\") pod \"authorino-58cd955675-q65tg\" (UID: \"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5\") " pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:53.749459 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.749352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrxsc\" (UniqueName: \"kubernetes.io/projected/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5-kube-api-access-jrxsc\") pod \"authorino-58cd955675-q65tg\" (UID: \"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5\") " pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:53.802805 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:53.802568    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:54.069444 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:54.069399    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-q65tg"]
Jul 29 19:38:54.074323 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:38:54.074289    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f00bcab_2c20_40ad_ac23_4349a9f6f4e5.slice/crio-7cbf8eef97c240f3ad04170e5cdffcd4061011b8c214c5eb6d14b43bea45797f WatchSource:0}: Error finding container 7cbf8eef97c240f3ad04170e5cdffcd4061011b8c214c5eb6d14b43bea45797f: Status 404 returned error can't find the container with id 7cbf8eef97c240f3ad04170e5cdffcd4061011b8c214c5eb6d14b43bea45797f
Jul 29 19:38:54.569282 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:54.569243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q65tg" event={"ID":"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5","Type":"ContainerStarted","Data":"7cbf8eef97c240f3ad04170e5cdffcd4061011b8c214c5eb6d14b43bea45797f"}
Jul 29 19:38:58.069258 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.069231    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q65tg"]
Jul 29 19:38:58.598576 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.598531    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl" event={"ID":"6c077cdc-8304-4e46-b45b-aa15fde400bf","Type":"ContainerStarted","Data":"0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688"}
Jul 29 19:38:58.598763 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.598698    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:38:58.600010 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.599961    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q65tg" event={"ID":"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5","Type":"ContainerStarted","Data":"0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef"}
Jul 29 19:38:58.600010 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.599979    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-q65tg" podUID="9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" containerName="authorino" containerID="cri-o://0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef" gracePeriod=30
Jul 29 19:38:58.601472 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.601450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg" event={"ID":"56c9f00e-289b-4e20-a364-c4a189c6c469","Type":"ContainerStarted","Data":"1dfb4f974f852512cc6629b9bc39bb74493ed2c19167e28b163bb5de6c97dc09"}
Jul 29 19:38:58.601650 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.601631    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:38:58.619834 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.619798    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl" podStartSLOduration=1.658856128 podStartE2EDuration="6.619784259s" totalImagesPullingTime="4.960928131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:58.618458462 +0000 UTC m=+560.496312789" watchObservedRunningTime="2026-07-29 19:38:58.619784259 +0000 UTC m=+560.497638609"
Jul 29 19:38:58.641204 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.641151    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg" podStartSLOduration=1.82548346 podStartE2EDuration="6.641133375s" totalImagesPullingTime="4.815649915s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:58.640043713 +0000 UTC m=+560.517898041" watchObservedRunningTime="2026-07-29 19:38:58.641133375 +0000 UTC m=+560.518987704"
Jul 29 19:38:58.660277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.660236    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-q65tg" podStartSLOduration=1.748055635 podStartE2EDuration="5.660223787s" totalImagesPullingTime="3.912168152s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:38:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:38:58.659428562 +0000 UTC m=+560.537282890" watchObservedRunningTime="2026-07-29 19:38:58.660223787 +0000 UTC m=+560.538078114"
Jul 29 19:38:58.857805 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.857752    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:58.984689 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.984660    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jrxsc\" (UniqueName: \"kubernetes.io/projected/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5-kube-api-access-jrxsc\") pod \"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5\" (UID: \"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5\") "
Jul 29 19:38:58.986891 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:58.986869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5-kube-api-access-jrxsc" (OuterVolumeSpecName: "kube-api-access-jrxsc") pod "9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" (UID: "9f00bcab-2c20-40ad-ac23-4349a9f6f4e5"). InnerVolumeSpecName "kube-api-access-jrxsc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:38:59.090741 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.086479    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jrxsc\" (UniqueName: \"kubernetes.io/projected/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5-kube-api-access-jrxsc\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:38:59.608008 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.607937    2580 generic.go:356] "Generic (PLEG): container finished" podID="9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" containerID="0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef" exitCode=0
Jul 29 19:38:59.608184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.608024    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-q65tg"
Jul 29 19:38:59.608184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.608018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q65tg" event={"ID":"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5","Type":"ContainerDied","Data":"0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef"}
Jul 29 19:38:59.608184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.608139    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-q65tg" event={"ID":"9f00bcab-2c20-40ad-ac23-4349a9f6f4e5","Type":"ContainerDied","Data":"7cbf8eef97c240f3ad04170e5cdffcd4061011b8c214c5eb6d14b43bea45797f"}
Jul 29 19:38:59.608184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.608157    2580 scope.go:117] "RemoveContainer" containerID="0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef"
Jul 29 19:38:59.617733 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.617712    2580 scope.go:117] "RemoveContainer" containerID="0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef"
Jul 29 19:38:59.618083 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:38:59.618057    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef\": container with ID starting with 0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef not found: ID does not exist" containerID="0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef"
Jul 29 19:38:59.618141 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.618097    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef"} err="failed to get container status \"0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef\": rpc error: code = NotFound desc = could not find container \"0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef\": container with ID starting with 0607382e6a61023234a0c8785717cb9822b65286c7790a78d7b239eb84e4c7ef not found: ID does not exist"
Jul 29 19:38:59.630735 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.630714    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-q65tg"]
Jul 29 19:38:59.634833 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:38:59.634814    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-q65tg"]
Jul 29 19:39:00.093591 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:00.093554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:00.717365 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:00.717328    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" path="/var/lib/kubelet/pods/9f00bcab-2c20-40ad-ac23-4349a9f6f4e5/volumes"
Jul 29 19:39:09.609983 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:09.609949    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"
Jul 29 19:39:09.611415 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:09.611393    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:39:09.694824 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:09.694783    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:39:09.695057 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:09.695031    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl" podUID="6c077cdc-8304-4e46-b45b-aa15fde400bf" containerName="limitador" containerID="cri-o://0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688" gracePeriod=30
Jul 29 19:39:10.273860 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.273833    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:39:10.397882 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.397857    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2xv8\" (UniqueName: \"kubernetes.io/projected/6c077cdc-8304-4e46-b45b-aa15fde400bf-kube-api-access-g2xv8\") pod \"6c077cdc-8304-4e46-b45b-aa15fde400bf\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") "
Jul 29 19:39:10.398050 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.397907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c077cdc-8304-4e46-b45b-aa15fde400bf-config-file\") pod \"6c077cdc-8304-4e46-b45b-aa15fde400bf\" (UID: \"6c077cdc-8304-4e46-b45b-aa15fde400bf\") "
Jul 29 19:39:10.398259 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.398234    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c077cdc-8304-4e46-b45b-aa15fde400bf-config-file" (OuterVolumeSpecName: "config-file") pod "6c077cdc-8304-4e46-b45b-aa15fde400bf" (UID: "6c077cdc-8304-4e46-b45b-aa15fde400bf"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 19:39:10.400093 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.400065    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c077cdc-8304-4e46-b45b-aa15fde400bf-kube-api-access-g2xv8" (OuterVolumeSpecName: "kube-api-access-g2xv8") pod "6c077cdc-8304-4e46-b45b-aa15fde400bf" (UID: "6c077cdc-8304-4e46-b45b-aa15fde400bf"). InnerVolumeSpecName "kube-api-access-g2xv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:39:10.501325 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.498905    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6c077cdc-8304-4e46-b45b-aa15fde400bf-config-file\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:39:10.501325 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.498947    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2xv8\" (UniqueName: \"kubernetes.io/projected/6c077cdc-8304-4e46-b45b-aa15fde400bf-kube-api-access-g2xv8\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:39:10.672444 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.672411    2580 generic.go:356] "Generic (PLEG): container finished" podID="6c077cdc-8304-4e46-b45b-aa15fde400bf" containerID="0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688" exitCode=0
Jul 29 19:39:10.672881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.672469    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"
Jul 29 19:39:10.672881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.672473    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl" event={"ID":"6c077cdc-8304-4e46-b45b-aa15fde400bf","Type":"ContainerDied","Data":"0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688"}
Jul 29 19:39:10.672881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.672630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-vd8pl" event={"ID":"6c077cdc-8304-4e46-b45b-aa15fde400bf","Type":"ContainerDied","Data":"05fe7e031f566d1424d7071017f8cd3deeb5b97d2ca604279ac2dd4a567fc8e7"}
Jul 29 19:39:10.672881 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.672664    2580 scope.go:117] "RemoveContainer" containerID="0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688"
Jul 29 19:39:10.682318 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.682296    2580 scope.go:117] "RemoveContainer" containerID="0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688"
Jul 29 19:39:10.682618 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:39:10.682597    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688\": container with ID starting with 0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688 not found: ID does not exist" containerID="0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688"
Jul 29 19:39:10.682673 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.682627    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688"} err="failed to get container status \"0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688\": rpc error: code = NotFound desc = could not find container \"0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688\": container with ID starting with 0d402b5e88524fe045a04f77892b500201dd105caccf719eaff36985cb432688 not found: ID does not exist"
Jul 29 19:39:10.696168 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.696128    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:39:10.698779 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.698751    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-vd8pl"]
Jul 29 19:39:10.715889 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:10.715863    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c077cdc-8304-4e46-b45b-aa15fde400bf" path="/var/lib/kubelet/pods/6c077cdc-8304-4e46-b45b-aa15fde400bf/volumes"
Jul 29 19:39:13.735018 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:13.734974    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:13.844741 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:13.844710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:14.123984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.123905    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"]
Jul 29 19:39:14.124338 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.124325    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c077cdc-8304-4e46-b45b-aa15fde400bf" containerName="limitador"
Jul 29 19:39:14.124338 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.124339    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c077cdc-8304-4e46-b45b-aa15fde400bf" containerName="limitador"
Jul 29 19:39:14.124432 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.124350    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" containerName="authorino"
Jul 29 19:39:14.124432 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.124356    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" containerName="authorino"
Jul 29 19:39:14.124511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.124434    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f00bcab-2c20-40ad-ac23-4349a9f6f4e5" containerName="authorino"
Jul 29 19:39:14.124511 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.124443    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c077cdc-8304-4e46-b45b-aa15fde400bf" containerName="limitador"
Jul 29 19:39:14.129175 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.129154    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.131031 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.130982    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hkx4p\""
Jul 29 19:39:14.131031 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.131027    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 29 19:39:14.131456 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.131437    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 29 19:39:14.131549 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.131477    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Jul 29 19:39:14.140845 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.140819    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"]
Jul 29 19:39:14.157406 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.157375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:14.232455 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.232409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/272e1ad1-770c-4d30-9942-3b378ea28713-data\") pod \"postgres-69fb4594cf-lj9jb\" (UID: \"272e1ad1-770c-4d30-9942-3b378ea28713\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.232640 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.232472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sxhl\" (UniqueName: \"kubernetes.io/projected/272e1ad1-770c-4d30-9942-3b378ea28713-kube-api-access-6sxhl\") pod \"postgres-69fb4594cf-lj9jb\" (UID: \"272e1ad1-770c-4d30-9942-3b378ea28713\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.333218 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.333177    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/272e1ad1-770c-4d30-9942-3b378ea28713-data\") pod \"postgres-69fb4594cf-lj9jb\" (UID: \"272e1ad1-770c-4d30-9942-3b378ea28713\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.333218 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.333221    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sxhl\" (UniqueName: \"kubernetes.io/projected/272e1ad1-770c-4d30-9942-3b378ea28713-kube-api-access-6sxhl\") pod \"postgres-69fb4594cf-lj9jb\" (UID: \"272e1ad1-770c-4d30-9942-3b378ea28713\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.333582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.333561    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/272e1ad1-770c-4d30-9942-3b378ea28713-data\") pod \"postgres-69fb4594cf-lj9jb\" (UID: \"272e1ad1-770c-4d30-9942-3b378ea28713\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.343191 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.343162    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sxhl\" (UniqueName: \"kubernetes.io/projected/272e1ad1-770c-4d30-9942-3b378ea28713-kube-api-access-6sxhl\") pod \"postgres-69fb4594cf-lj9jb\" (UID: \"272e1ad1-770c-4d30-9942-3b378ea28713\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.439888 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.439852    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:14.593831 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.593805    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"]
Jul 29 19:39:14.596055 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:39:14.596030    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod272e1ad1_770c_4d30_9942_3b378ea28713.slice/crio-59a6b2f0fe3317e822327afa5254029f617a5c0f97872bbcfba9ec35901a5b33 WatchSource:0}: Error finding container 59a6b2f0fe3317e822327afa5254029f617a5c0f97872bbcfba9ec35901a5b33: Status 404 returned error can't find the container with id 59a6b2f0fe3317e822327afa5254029f617a5c0f97872bbcfba9ec35901a5b33
Jul 29 19:39:14.696214 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:14.696125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb" event={"ID":"272e1ad1-770c-4d30-9942-3b378ea28713","Type":"ContainerStarted","Data":"59a6b2f0fe3317e822327afa5254029f617a5c0f97872bbcfba9ec35901a5b33"}
Jul 29 19:39:20.735345 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:20.735305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb" event={"ID":"272e1ad1-770c-4d30-9942-3b378ea28713","Type":"ContainerStarted","Data":"b743cc806aea6b8e731686763e902fd19956ff16f56a9915d33707dca69de969"}
Jul 29 19:39:20.735827 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:20.735424    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:20.753730 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:20.753682    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb" podStartSLOduration=1.659543706 podStartE2EDuration="6.753666917s" totalImagesPullingTime="5.094123211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:39:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:39:20.752547645 +0000 UTC m=+582.630401974" watchObservedRunningTime="2026-07-29 19:39:20.753666917 +0000 UTC m=+582.631521245"
Jul 29 19:39:26.771446 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:26.771418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-lj9jb"
Jul 29 19:39:27.085294 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.085210    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:27.332376 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.332338    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bj2pb"]
Jul 29 19:39:27.340853 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.340774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:27.341556 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.341534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bj2pb"]
Jul 29 19:39:27.342430 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.342414    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ldhdt\""
Jul 29 19:39:27.348133 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.348107    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:27.449933 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.449894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm7lj\" (UniqueName: \"kubernetes.io/projected/8aa0f471-a1ba-4dab-a42a-051008c5ccd6-kube-api-access-fm7lj\") pod \"authorino-5d4b67c446-bj2pb\" (UID: \"8aa0f471-a1ba-4dab-a42a-051008c5ccd6\") " pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:27.551204 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.551167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm7lj\" (UniqueName: \"kubernetes.io/projected/8aa0f471-a1ba-4dab-a42a-051008c5ccd6-kube-api-access-fm7lj\") pod \"authorino-5d4b67c446-bj2pb\" (UID: \"8aa0f471-a1ba-4dab-a42a-051008c5ccd6\") " pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:27.562597 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.562564    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bj2pb"]
Jul 29 19:39:27.562887 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:39:27.562867    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-fm7lj], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-bj2pb" podUID="8aa0f471-a1ba-4dab-a42a-051008c5ccd6"
Jul 29 19:39:27.564009 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.563967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm7lj\" (UniqueName: \"kubernetes.io/projected/8aa0f471-a1ba-4dab-a42a-051008c5ccd6-kube-api-access-fm7lj\") pod \"authorino-5d4b67c446-bj2pb\" (UID: \"8aa0f471-a1ba-4dab-a42a-051008c5ccd6\") " pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:27.588429 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.588400    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59d8b4fbbb-hbdf5"]
Jul 29 19:39:27.592402 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.592346    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.594439 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.594418    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Jul 29 19:39:27.603371 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.603349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59d8b4fbbb-hbdf5"]
Jul 29 19:39:27.652246 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.652214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1c11c2e-dc24-4059-90cb-741517eac563-tls-cert\") pod \"authorino-59d8b4fbbb-hbdf5\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") " pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.652425 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.652261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttf4j\" (UniqueName: \"kubernetes.io/projected/e1c11c2e-dc24-4059-90cb-741517eac563-kube-api-access-ttf4j\") pod \"authorino-59d8b4fbbb-hbdf5\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") " pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.753617 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.753576    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1c11c2e-dc24-4059-90cb-741517eac563-tls-cert\") pod \"authorino-59d8b4fbbb-hbdf5\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") " pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.753801 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.753642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttf4j\" (UniqueName: \"kubernetes.io/projected/e1c11c2e-dc24-4059-90cb-741517eac563-kube-api-access-ttf4j\") pod \"authorino-59d8b4fbbb-hbdf5\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") " pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.756334 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.756313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1c11c2e-dc24-4059-90cb-741517eac563-tls-cert\") pod \"authorino-59d8b4fbbb-hbdf5\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") " pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.762624 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.762595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttf4j\" (UniqueName: \"kubernetes.io/projected/e1c11c2e-dc24-4059-90cb-741517eac563-kube-api-access-ttf4j\") pod \"authorino-59d8b4fbbb-hbdf5\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") " pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.773351 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.773330    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:27.780764 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.780746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:27.855110 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.855031    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fm7lj\" (UniqueName: \"kubernetes.io/projected/8aa0f471-a1ba-4dab-a42a-051008c5ccd6-kube-api-access-fm7lj\") pod \"8aa0f471-a1ba-4dab-a42a-051008c5ccd6\" (UID: \"8aa0f471-a1ba-4dab-a42a-051008c5ccd6\") "
Jul 29 19:39:27.857292 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.857265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8aa0f471-a1ba-4dab-a42a-051008c5ccd6-kube-api-access-fm7lj" (OuterVolumeSpecName: "kube-api-access-fm7lj") pod "8aa0f471-a1ba-4dab-a42a-051008c5ccd6" (UID: "8aa0f471-a1ba-4dab-a42a-051008c5ccd6"). InnerVolumeSpecName "kube-api-access-fm7lj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:39:27.902508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.902459    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:39:27.956736 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:27.956664    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fm7lj\" (UniqueName: \"kubernetes.io/projected/8aa0f471-a1ba-4dab-a42a-051008c5ccd6-kube-api-access-fm7lj\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:39:28.038238 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.038213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59d8b4fbbb-hbdf5"]
Jul 29 19:39:28.040574 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:39:28.040550    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1c11c2e_dc24_4059_90cb_741517eac563.slice/crio-5a2355e9964e4dbf177ebdc7b701b7fce15f49149b0da933a4ef1d751deee259 WatchSource:0}: Error finding container 5a2355e9964e4dbf177ebdc7b701b7fce15f49149b0da933a4ef1d751deee259: Status 404 returned error can't find the container with id 5a2355e9964e4dbf177ebdc7b701b7fce15f49149b0da933a4ef1d751deee259
Jul 29 19:39:28.779330 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.779247    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-bj2pb"
Jul 29 19:39:28.779330 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.779297    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5" event={"ID":"e1c11c2e-dc24-4059-90cb-741517eac563","Type":"ContainerStarted","Data":"bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1"}
Jul 29 19:39:28.779785 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.779342    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5" event={"ID":"e1c11c2e-dc24-4059-90cb-741517eac563","Type":"ContainerStarted","Data":"5a2355e9964e4dbf177ebdc7b701b7fce15f49149b0da933a4ef1d751deee259"}
Jul 29 19:39:28.797947 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.797892    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5" podStartSLOduration=1.342128569 podStartE2EDuration="1.797877224s" totalImagesPullingTime="455.748655ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:39:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:39:28.79532854 +0000 UTC m=+590.673182869" watchObservedRunningTime="2026-07-29 19:39:28.797877224 +0000 UTC m=+590.675731549"
Jul 29 19:39:28.831842 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.831809    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bj2pb"]
Jul 29 19:39:28.840382 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:28.840352    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-bj2pb"]
Jul 29 19:39:30.717256 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:30.717224    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8aa0f471-a1ba-4dab-a42a-051008c5ccd6" path="/var/lib/kubelet/pods/8aa0f471-a1ba-4dab-a42a-051008c5ccd6/volumes"
Jul 29 19:39:33.447908 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:33.447871    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:33.535846 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:33.535808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:33.628013 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:33.627968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:33.797894 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:33.797806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:36.773935 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:36.773900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:36.879051 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:36.878970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:39:38.586266 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:38.586236    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:39:38.591802 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:38.591777    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:39:38.593303 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:38.593282    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:39:38.598586 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:39:38.598568    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:40:08.379336 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:08.379293    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:09.816662 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:09.816626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:09.915289 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:09.915259    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:09.948567 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:09.948528    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:40:18.056657 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:18.056625    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f016396-42e6-4348-a688-dc65be237fa3" containerID="8b466fc4ad314236bab58ad4a36720603001d9a549afa48571d5576a6ec9b510" exitCode=0
Jul 29 19:40:18.057201 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:18.056708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" event={"ID":"4f016396-42e6-4348-a688-dc65be237fa3","Type":"ContainerDied","Data":"8b466fc4ad314236bab58ad4a36720603001d9a549afa48571d5576a6ec9b510"}
Jul 29 19:40:18.057267 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:18.057249    2580 scope.go:117] "RemoveContainer" containerID="8b466fc4ad314236bab58ad4a36720603001d9a549afa48571d5576a6ec9b510"
Jul 29 19:40:18.058307 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:18.058273    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:40:18.058411 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:18.058356    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:40:18.058411 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:18.058402    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 19:40:19.063215 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:19.063178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" event={"ID":"4f016396-42e6-4348-a688-dc65be237fa3","Type":"ContainerStarted","Data":"e3d7ad10b93c853906075ad266a16aa7a7429d1c79c319f59991e05b95f7c5ac"}
Jul 29 19:40:19.335684 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:19.335586    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:19.432932 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:19.432853    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:19.575982 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:19.575947    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:40:19.577273 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:19.577247    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-7rzcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 29 19:40:19.577385 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:19.577298    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" podUID="4f016396-42e6-4348-a688-dc65be237fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 29 19:40:20.576384 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:20.576344    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-7rzcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 29 19:40:20.576790 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:20.576404    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" podUID="4f016396-42e6-4348-a688-dc65be237fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 29 19:40:21.576346 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:21.576307    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-7rzcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 29 19:40:21.576508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:21.576366    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" podUID="4f016396-42e6-4348-a688-dc65be237fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 29 19:40:22.552984 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:22.552944    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:40:22.575799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:22.575762    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-7rzcx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body=
Jul 29 19:40:22.575969 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:22.575830    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx" podUID="4f016396-42e6-4348-a688-dc65be237fa3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused"
Jul 29 19:40:23.580080 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:23.580048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:40:23.580952 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:23.580932    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7rzcx"
Jul 29 19:40:25.541493 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:25.541459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:26.020332 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:26.020293    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:40:30.810582 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:30.810540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:31.897709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:31.897671    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:40:41.824039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:41.823977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:43.103306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:43.103266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:40:53.875845 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.875812    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"]
Jul 29 19:40:53.878799 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.878779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:53.880848 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.880819    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 29 19:40:53.881279 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.881260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-8fhrs\""
Jul 29 19:40:53.881391 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.881275    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 29 19:40:53.881391 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.881261    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 29 19:40:53.889983 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.889963    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"]
Jul 29 19:40:53.911322 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.911295    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:40:53.981747 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.981701    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:53.981929 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.981764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:53.981929 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.981815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c1cbe5d-ba5a-42da-82a2-f4355876d802-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:53.981929 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.981874    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjtbc\" (UniqueName: \"kubernetes.io/projected/6c1cbe5d-ba5a-42da-82a2-f4355876d802-kube-api-access-tjtbc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:53.981929 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.981901    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:53.982110 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:53.981963    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083218 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083186    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083416 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083416 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083416 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083300    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c1cbe5d-ba5a-42da-82a2-f4355876d802-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083416 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjtbc\" (UniqueName: \"kubernetes.io/projected/6c1cbe5d-ba5a-42da-82a2-f4355876d802-kube-api-access-tjtbc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083416 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083358    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083685 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083685 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083677    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.083786 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.083714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.085762 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.085732    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6c1cbe5d-ba5a-42da-82a2-f4355876d802-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.085904 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.085884    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6c1cbe5d-ba5a-42da-82a2-f4355876d802-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.092335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.092308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjtbc\" (UniqueName: \"kubernetes.io/projected/6c1cbe5d-ba5a-42da-82a2-f4355876d802-kube-api-access-tjtbc\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx\" (UID: \"6c1cbe5d-ba5a-42da-82a2-f4355876d802\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.190778 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.190681    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:40:54.333137 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.333112    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"]
Jul 29 19:40:54.335127 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:40:54.335100    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c1cbe5d_ba5a_42da_82a2_f4355876d802.slice/crio-db5a4c06dc2ba434697ad5163d421f3be5a3609af6a0b88a276a4b2badb46687 WatchSource:0}: Error finding container db5a4c06dc2ba434697ad5163d421f3be5a3609af6a0b88a276a4b2badb46687: Status 404 returned error can't find the container with id db5a4c06dc2ba434697ad5163d421f3be5a3609af6a0b88a276a4b2badb46687
Jul 29 19:40:54.336473 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:54.336456    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 19:40:55.264553 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:55.264516    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx" event={"ID":"6c1cbe5d-ba5a-42da-82a2-f4355876d802","Type":"ContainerStarted","Data":"db5a4c06dc2ba434697ad5163d421f3be5a3609af6a0b88a276a4b2badb46687"}
Jul 29 19:40:55.598657 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:55.598579    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:40:56.272521 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:56.272475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx" event={"ID":"6c1cbe5d-ba5a-42da-82a2-f4355876d802","Type":"ContainerStarted","Data":"c4a6add0379fe2d6aa9389babf699deec79e34b04e02a3ab9ea81367c5b67806"}
Jul 29 19:40:56.296903 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:40:56.296850    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx" podStartSLOduration=2.3455198839999998 podStartE2EDuration="3.296831897s" totalImagesPullingTime="951.312013ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:40:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:40:56.294872071 +0000 UTC m=+678.172726399" watchObservedRunningTime="2026-07-29 19:40:56.296831897 +0000 UTC m=+678.174686225"
Jul 29 19:41:04.191438 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:04.191399    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:41:04.191988 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:04.191484    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:41:04.204179 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:04.204155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:41:04.326142 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:04.326115    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx"
Jul 29 19:41:07.377361 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.377329    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"]
Jul 29 19:41:07.380559 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.380537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.382303 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.382284    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 29 19:41:07.391039 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.390986    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"]
Jul 29 19:41:07.400487 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.400455    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:41:07.514446 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.514408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.514446 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.514453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.514716 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.514485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbkc6\" (UniqueName: \"kubernetes.io/projected/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-kube-api-access-tbkc6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.514716 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.514571    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.514716 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.514609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.514716 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.514638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.615978 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.615941    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616018    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbkc6\" (UniqueName: \"kubernetes.io/projected/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-kube-api-access-tbkc6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616191    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616417 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616398    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616533 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616411    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.616629 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.616612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.618679 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.618649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.618976 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.618958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.626680 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.626653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbkc6\" (UniqueName: \"kubernetes.io/projected/a36710e8-c8b8-4a86-93c1-1a1defa6fda0-kube-api-access-tbkc6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-gnp4s\" (UID: \"a36710e8-c8b8-4a86-93c1-1a1defa6fda0\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.691390 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.691297    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:07.830313 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:07.830253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"]
Jul 29 19:41:07.833322 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:41:07.833296    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda36710e8_c8b8_4a86_93c1_1a1defa6fda0.slice/crio-66f1e19fbfd6875f19a933db2d2051ff2783ff2176fc672776f731682b88a23d WatchSource:0}: Error finding container 66f1e19fbfd6875f19a933db2d2051ff2783ff2176fc672776f731682b88a23d: Status 404 returned error can't find the container with id 66f1e19fbfd6875f19a933db2d2051ff2783ff2176fc672776f731682b88a23d
Jul 29 19:41:08.339185 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:08.339087    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s" event={"ID":"a36710e8-c8b8-4a86-93c1-1a1defa6fda0","Type":"ContainerStarted","Data":"efe56701dbc4da9aafb08f91321480b983dbfacd73c90564c52b80d0f11506bd"}
Jul 29 19:41:08.339185 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:08.339147    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s" event={"ID":"a36710e8-c8b8-4a86-93c1-1a1defa6fda0","Type":"ContainerStarted","Data":"66f1e19fbfd6875f19a933db2d2051ff2783ff2176fc672776f731682b88a23d"}
Jul 29 19:41:08.358406 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:08.358352    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s" podStartSLOduration=1.176140473 podStartE2EDuration="1.358333561s" totalImagesPullingTime="182.193088ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:41:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:41:08.356511427 +0000 UTC m=+690.234365755" watchObservedRunningTime="2026-07-29 19:41:08.358333561 +0000 UTC m=+690.236187891"
Jul 29 19:41:10.205134 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:10.205091    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:41:17.691592 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:17.691552    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:17.692038 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:17.691614    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:17.704311 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:17.704285    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:18.404469 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:18.404438    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-gnp4s"
Jul 29 19:41:46.273764 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:46.273674    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:41:48.598302 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:41:48.598267    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:42:14.223277 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:42:14.223242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:42:28.468877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:42:28.468838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:42:41.227448 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:42:41.227411    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:04.740636 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.740555    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-bbcb9d87-8pf22"]
Jul 29 19:43:04.744117 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.744097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:04.749649 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.749618    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bbcb9d87-8pf22"]
Jul 29 19:43:04.869355 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.869314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2665e292-089d-441e-b40d-aa42429ec3fc-tls-cert\") pod \"authorino-bbcb9d87-8pf22\" (UID: \"2665e292-089d-441e-b40d-aa42429ec3fc\") " pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:04.869556 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.869473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgcnd\" (UniqueName: \"kubernetes.io/projected/2665e292-089d-441e-b40d-aa42429ec3fc-kube-api-access-xgcnd\") pod \"authorino-bbcb9d87-8pf22\" (UID: \"2665e292-089d-441e-b40d-aa42429ec3fc\") " pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:04.970301 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.970265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2665e292-089d-441e-b40d-aa42429ec3fc-tls-cert\") pod \"authorino-bbcb9d87-8pf22\" (UID: \"2665e292-089d-441e-b40d-aa42429ec3fc\") " pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:04.970496 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.970393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgcnd\" (UniqueName: \"kubernetes.io/projected/2665e292-089d-441e-b40d-aa42429ec3fc-kube-api-access-xgcnd\") pod \"authorino-bbcb9d87-8pf22\" (UID: \"2665e292-089d-441e-b40d-aa42429ec3fc\") " pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:04.973014 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.972970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2665e292-089d-441e-b40d-aa42429ec3fc-tls-cert\") pod \"authorino-bbcb9d87-8pf22\" (UID: \"2665e292-089d-441e-b40d-aa42429ec3fc\") " pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:04.978389 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:04.978361    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgcnd\" (UniqueName: \"kubernetes.io/projected/2665e292-089d-441e-b40d-aa42429ec3fc-kube-api-access-xgcnd\") pod \"authorino-bbcb9d87-8pf22\" (UID: \"2665e292-089d-441e-b40d-aa42429ec3fc\") " pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:05.054740 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:05.054640    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-bbcb9d87-8pf22"
Jul 29 19:43:05.148142 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:05.148111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:05.199624 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:05.199599    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-bbcb9d87-8pf22"]
Jul 29 19:43:05.201543 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:43:05.201517    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2665e292_089d_441e_b40d_aa42429ec3fc.slice/crio-ad2f8f094a0d563893c9ac75f1dd91b3af1c532eb255f39c482956cf2294a883 WatchSource:0}: Error finding container ad2f8f094a0d563893c9ac75f1dd91b3af1c532eb255f39c482956cf2294a883: Status 404 returned error can't find the container with id ad2f8f094a0d563893c9ac75f1dd91b3af1c532eb255f39c482956cf2294a883
Jul 29 19:43:05.247468 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:05.247438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:05.988059 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:05.987945    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bbcb9d87-8pf22" event={"ID":"2665e292-089d-441e-b40d-aa42429ec3fc","Type":"ContainerStarted","Data":"a55b2b99972106e3d2b33dbb3aac41900bcab45ba3934b17200e420eec8ac0ae"}
Jul 29 19:43:05.988059 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:05.988017    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-bbcb9d87-8pf22" event={"ID":"2665e292-089d-441e-b40d-aa42429ec3fc","Type":"ContainerStarted","Data":"ad2f8f094a0d563893c9ac75f1dd91b3af1c532eb255f39c482956cf2294a883"}
Jul 29 19:43:06.001619 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.001541    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-bbcb9d87-8pf22" podStartSLOduration=1.480938001 podStartE2EDuration="2.001516546s" totalImagesPullingTime="520.578545ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 19:43:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 19:43:06.00077947 +0000 UTC m=+807.878633831" watchObservedRunningTime="2026-07-29 19:43:06.001516546 +0000 UTC m=+807.879370877"
Jul 29 19:43:06.024762 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.024725    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59d8b4fbbb-hbdf5"]
Jul 29 19:43:06.025020 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.024972    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5" podUID="e1c11c2e-dc24-4059-90cb-741517eac563" containerName="authorino" containerID="cri-o://bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1" gracePeriod=30
Jul 29 19:43:06.226505 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.226366    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:06.301117 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.301065    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:43:06.330910 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.330868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:06.385570 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.385533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttf4j\" (UniqueName: \"kubernetes.io/projected/e1c11c2e-dc24-4059-90cb-741517eac563-kube-api-access-ttf4j\") pod \"e1c11c2e-dc24-4059-90cb-741517eac563\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") "
Jul 29 19:43:06.385731 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.385618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1c11c2e-dc24-4059-90cb-741517eac563-tls-cert\") pod \"e1c11c2e-dc24-4059-90cb-741517eac563\" (UID: \"e1c11c2e-dc24-4059-90cb-741517eac563\") "
Jul 29 19:43:06.387864 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.387823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1c11c2e-dc24-4059-90cb-741517eac563-kube-api-access-ttf4j" (OuterVolumeSpecName: "kube-api-access-ttf4j") pod "e1c11c2e-dc24-4059-90cb-741517eac563" (UID: "e1c11c2e-dc24-4059-90cb-741517eac563"). InnerVolumeSpecName "kube-api-access-ttf4j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 19:43:06.397731 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.397702    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1c11c2e-dc24-4059-90cb-741517eac563-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e1c11c2e-dc24-4059-90cb-741517eac563" (UID: "e1c11c2e-dc24-4059-90cb-741517eac563"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 19:43:06.486651 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.486612    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e1c11c2e-dc24-4059-90cb-741517eac563-tls-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:43:06.486651 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.486646    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttf4j\" (UniqueName: \"kubernetes.io/projected/e1c11c2e-dc24-4059-90cb-741517eac563-kube-api-access-ttf4j\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 19:43:06.994603 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.994569    2580 generic.go:356] "Generic (PLEG): container finished" podID="e1c11c2e-dc24-4059-90cb-741517eac563" containerID="bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1" exitCode=0
Jul 29 19:43:06.995058 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.994609    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5"
Jul 29 19:43:06.995058 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.994650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5" event={"ID":"e1c11c2e-dc24-4059-90cb-741517eac563","Type":"ContainerDied","Data":"bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1"}
Jul 29 19:43:06.995058 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.994691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59d8b4fbbb-hbdf5" event={"ID":"e1c11c2e-dc24-4059-90cb-741517eac563","Type":"ContainerDied","Data":"5a2355e9964e4dbf177ebdc7b701b7fce15f49149b0da933a4ef1d751deee259"}
Jul 29 19:43:06.995058 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:06.994709    2580 scope.go:117] "RemoveContainer" containerID="bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1"
Jul 29 19:43:07.004131 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:07.004110    2580 scope.go:117] "RemoveContainer" containerID="bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1"
Jul 29 19:43:07.004435 ip-10-0-140-87 kubenswrapper[2580]: E0729 19:43:07.004405    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1\": container with ID starting with bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1 not found: ID does not exist" containerID="bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1"
Jul 29 19:43:07.004506 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:07.004445    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1"} err="failed to get container status \"bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1\": rpc error: code = NotFound desc = could not find container \"bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1\": container with ID starting with bc73a1a9b992fa3a6f9806e8f398b1ebefc271ac1e3967750fcc5b504ea776b1 not found: ID does not exist"
Jul 29 19:43:07.009939 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:07.009916    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59d8b4fbbb-hbdf5"]
Jul 29 19:43:07.016070 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:07.016040    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59d8b4fbbb-hbdf5"]
Jul 29 19:43:08.717194 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:08.717154    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1c11c2e-dc24-4059-90cb-741517eac563" path="/var/lib/kubelet/pods/e1c11c2e-dc24-4059-90cb-741517eac563/volumes"
Jul 29 19:43:16.138610 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:16.138569    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:16.193498 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:16.193470    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:28.343335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:28.343294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:43:32.998306 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:32.998264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:43:43.995658 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:43.995623    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:43:53.406396 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:43:53.406362    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:44:38.622625 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:44:38.622549    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:44:38.625316 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:44:38.625296    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:44:38.629249 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:44:38.629227    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:44:38.631627 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:44:38.631611    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:45:10.314912 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:10.314876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:11.746862 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:11.746768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:31.307046 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:31.307009    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:45:44.597202 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:44.597164    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:45:46.702969 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:46.702937    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:46.803121 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:46.803087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:46.917136 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:46.917105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:53.061816 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:53.061778    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:53.168759 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:53.168721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:53.262474 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:53.262445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:53.407117 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:53.407081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:59.180355 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:59.180317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:45:59.241329 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:45:59.241294    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:46:15.203357 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:46:15.203321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:46:23.095647 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:46:23.095607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:46:33.605842 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:46:33.605804    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:46:41.399730 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:46:41.399695    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:46:50.906739 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:46:50.906687    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:47:07.204054 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:07.204015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:47:13.595922 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:13.595879    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:47:21.605308 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:21.605263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:47:24.780176 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:24.780139    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:47:24.834570 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:24.834516    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:47:43.801501 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:43.801420    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:47:52.800946 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:47:52.800907    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:48:15.296300 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:48:15.296263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:48:26.807599 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:48:26.807559    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:48:36.095069 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:48:36.095033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:48:51.607359 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:48:51.607319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:49:00.805153 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:00.805051    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:49:09.998412 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:09.998371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:49:19.694828 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:19.694791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:49:28.298493 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:28.298462    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:49:38.666851 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:38.666820    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:49:38.667798 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:38.667760    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:49:38.673433 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:38.673410    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:49:38.674111 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:38.674093    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:49:40.795233 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:40.795198    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:49:48.691954 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:49:48.691916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:51:21.345086 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:51:21.345049    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:51:24.818857 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:51:24.818807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:51:28.299076 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:51:28.299035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:51:32.282701 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:51:32.282657    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:51:38.896508 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:51:38.896466    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:51:53.195260 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:51:53.195220    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:52:06.402252 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:52:06.402213    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:52:15.292675 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:52:15.292640    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:52:24.795955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:52:24.795916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:52:33.993945 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:52:33.993911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:52:43.302614 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:52:43.302575    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:52:54.700629 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:52:54.700589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:53:04.197335 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:53:04.197297    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:53:14.402883 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:53:14.402848    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:53:26.096843 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:53:26.096807    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:53:48.204674 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:53:48.204579    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:53:56.999716 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:53:56.999677    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:54:01.727585 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:01.727550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:54:11.912140 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:11.912050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:54:14.704766 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:14.704727    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:54:38.706052 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:38.706021    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:54:38.707253 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:38.707232    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:54:38.713674 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:38.713654    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:54:38.715184 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:38.715158    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:54:50.298739 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:50.298702    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:54:51.119379 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:51.119341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:54:53.595111 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:54:53.595072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:55:07.094813 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:55:07.094768    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:55:15.697043 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:55:15.697005    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:55:27.103026 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:55:27.102973    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:55:36.790563 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:55:36.790523    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:55:45.799957 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:55:45.799918    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:55:53.614055 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:55:53.614018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:56:02.800811 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:02.800768    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:56:12.411938 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:12.411901    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:56:13.650353 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:13.650316    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:56:13.753426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:13.753395    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:56:25.705304 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:25.705270    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:56:35.030533 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:35.030451    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:56:43.303412 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:43.303373    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:56:53.689638 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:56:53.689589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:57:05.594475 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:05.594440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:57:14.295737 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:14.295697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:57:25.706946 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:25.706903    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:57:37.603560 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:37.603521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:57:48.496314 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:48.496279    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:57:51.443536 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:51.443490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:57:56.909123 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:57:56.909088    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:58:10.384671 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:10.384627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:10.436426 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:10.436387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:14.973733 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:14.973699    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:17.930809 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:17.930772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:18.030224 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:18.030146    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:18.354671 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:18.354590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:18.607738 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:18.607660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:58:24.424550 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:24.424512    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:24.479831 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:24.479771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:24.654809 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:24.654773    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:24.974190 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:24.974153    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:25.028923 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:25.028884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:27.067047 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:27.066979    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:27.167248 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:27.167218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:27.267006 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:27.266960    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:27.499918 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:27.499846    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:27.777942 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:27.777866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:27.877300 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:27.877256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:30.496830 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:30.496793    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:58:57.899364 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:57.899315    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:57.997928 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:57.997892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.098915 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.098856    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.155488 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.155404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.272667 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.272635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.477606 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.477521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.702648 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.702602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.803730 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.803637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:58.901351 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:58.901297    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:59.201424 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:59.201370    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:58:59.429895 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:58:59.429862    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:01.384261 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:01.384213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:01.501204 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:01.501172    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:01.731714 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:01.731614    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:01.799815 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:01.799786    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:59:02.131474 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:02.131439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:02.233100 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:02.233067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:12.596788 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:12.596743    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:59:29.967214 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:29.967146    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:30.039758 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:30.039722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:30.099840 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:30.099793    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:30.192244 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:30.192206    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:30.327822 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:30.327734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:30.573877 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:30.573845    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:30.778326 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:30.778298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:31.090290 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:31.090200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:31.415921 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:31.415892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:33.425860 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:33.425816    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:33.513763 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:33.513725    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:33.608665 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:33.608625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:33.817315 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:33.817228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:34.002514 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:34.002481    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:59:34.133429 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:34.133397    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:34.233955 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:34.233919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 19:59:38.741243 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:38.741213    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:59:38.743291 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:38.743260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 19:59:38.747797 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:38.747776    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:59:38.749709 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:38.749691    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 19:59:44.199798 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:44.199756    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 19:59:59.208944 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.208909    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"]
Jul 29 19:59:59.209387 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.209372    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1c11c2e-dc24-4059-90cb-741517eac563" containerName="authorino"
Jul 29 19:59:59.209458 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.209388    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1c11c2e-dc24-4059-90cb-741517eac563" containerName="authorino"
Jul 29 19:59:59.209458 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.209457    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1c11c2e-dc24-4059-90cb-741517eac563" containerName="authorino"
Jul 29 19:59:59.212583 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.212566    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 19:59:59.214280 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.214260    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 19:59:59.220289 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.220269    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"]
Jul 29 19:59:59.330813 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.330776    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6695e9ba-6b47-4329-b73e-4fa7a5317c42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73e856f2-98hgl\" (UID: \"6695e9ba-6b47-4329-b73e-4fa7a5317c42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 19:59:59.432230 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.432190    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6695e9ba-6b47-4329-b73e-4fa7a5317c42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73e856f2-98hgl\" (UID: \"6695e9ba-6b47-4329-b73e-4fa7a5317c42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 19:59:59.432830 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.432809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6695e9ba-6b47-4329-b73e-4fa7a5317c42-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-73e856f2-98hgl\" (UID: \"6695e9ba-6b47-4329-b73e-4fa7a5317c42\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 19:59:59.524672 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.524590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 19:59:59.665176 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.665143    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"]
Jul 29 19:59:59.667391 ip-10-0-140-87 kubenswrapper[2580]: W0729 19:59:59.667360    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6695e9ba_6b47_4329_b73e_4fa7a5317c42.slice/crio-490792558b4c4496eace967910c86b4470a514a554c85886dd2f00b417bb32e8 WatchSource:0}: Error finding container 490792558b4c4496eace967910c86b4470a514a554c85886dd2f00b417bb32e8: Status 404 returned error can't find the container with id 490792558b4c4496eace967910c86b4470a514a554c85886dd2f00b417bb32e8
Jul 29 19:59:59.669356 ip-10-0-140-87 kubenswrapper[2580]: I0729 19:59:59.669333    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:00:00.131421 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.131388    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"]
Jul 29 20:00:00.136261 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.136241    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"
Jul 29 20:00:00.142013 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.141963    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"]
Jul 29 20:00:00.149169 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.149147    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"
Jul 29 20:00:00.492882 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.492849    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"]
Jul 29 20:00:00.494431 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:00:00.494399    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63c97a0a_169f_436e_9335_20480f664261.slice/crio-8a7dac2fc8026844c99891a7602c78743fb4bf75d779c08f8aa12ceae9f2ae76 WatchSource:0}: Error finding container 8a7dac2fc8026844c99891a7602c78743fb4bf75d779c08f8aa12ceae9f2ae76: Status 404 returned error can't find the container with id 8a7dac2fc8026844c99891a7602c78743fb4bf75d779c08f8aa12ceae9f2ae76
Jul 29 20:00:00.517931 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.517894    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b" event={"ID":"63c97a0a-169f-436e-9335-20480f664261","Type":"ContainerStarted","Data":"8a7dac2fc8026844c99891a7602c78743fb4bf75d779c08f8aa12ceae9f2ae76"}
Jul 29 20:00:00.519265 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:00.519239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl" event={"ID":"6695e9ba-6b47-4329-b73e-4fa7a5317c42","Type":"ContainerStarted","Data":"490792558b4c4496eace967910c86b4470a514a554c85886dd2f00b417bb32e8"}
Jul 29 20:00:05.556087 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:05.555974    2580 generic.go:356] "Generic (PLEG): container finished" podID="6695e9ba-6b47-4329-b73e-4fa7a5317c42" containerID="bf3d7718a77de5b647a9ba352ce3a4f7f9bf91f0e61f6abadce41df0c390c66b" exitCode=0
Jul 29 20:00:05.556087 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:05.556063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl" event={"ID":"6695e9ba-6b47-4329-b73e-4fa7a5317c42","Type":"ContainerDied","Data":"bf3d7718a77de5b647a9ba352ce3a4f7f9bf91f0e61f6abadce41df0c390c66b"}
Jul 29 20:00:05.557688 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:05.557667    2580 generic.go:356] "Generic (PLEG): container finished" podID="63c97a0a-169f-436e-9335-20480f664261" containerID="dde17656ed96f2ea7fae631667fa0d6d69eb43b18893b1489abe5006268f0c9f" exitCode=0
Jul 29 20:00:05.557795 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:05.557712    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b" event={"ID":"63c97a0a-169f-436e-9335-20480f664261","Type":"ContainerDied","Data":"dde17656ed96f2ea7fae631667fa0d6d69eb43b18893b1489abe5006268f0c9f"}
Jul 29 20:00:06.726680 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:06.726654    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"
Jul 29 20:00:06.729658 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:06.729637    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 20:00:06.898687 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:06.898655    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6695e9ba-6b47-4329-b73e-4fa7a5317c42-maas-api-service-ca\") pod \"6695e9ba-6b47-4329-b73e-4fa7a5317c42\" (UID: \"6695e9ba-6b47-4329-b73e-4fa7a5317c42\") "
Jul 29 20:00:06.899133 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:06.899102    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6695e9ba-6b47-4329-b73e-4fa7a5317c42-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6695e9ba-6b47-4329-b73e-4fa7a5317c42" (UID: "6695e9ba-6b47-4329-b73e-4fa7a5317c42"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:00:07.000400 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.000357    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6695e9ba-6b47-4329-b73e-4fa7a5317c42-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:00:07.577800 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.577768    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"
Jul 29 20:00:07.577985 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.577767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl" event={"ID":"6695e9ba-6b47-4329-b73e-4fa7a5317c42","Type":"ContainerDied","Data":"490792558b4c4496eace967910c86b4470a514a554c85886dd2f00b417bb32e8"}
Jul 29 20:00:07.577985 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.577888    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="490792558b4c4496eace967910c86b4470a514a554c85886dd2f00b417bb32e8"
Jul 29 20:00:07.579147 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.579122    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"
Jul 29 20:00:07.579252 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.579154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b" event={"ID":"63c97a0a-169f-436e-9335-20480f664261","Type":"ContainerDied","Data":"8a7dac2fc8026844c99891a7602c78743fb4bf75d779c08f8aa12ceae9f2ae76"}
Jul 29 20:00:07.579252 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:07.579185    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8a7dac2fc8026844c99891a7602c78743fb4bf75d779c08f8aa12ceae9f2ae76"
Jul 29 20:00:14.196764 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.196731    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"]
Jul 29 20:00:14.200335 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.200301    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-ait-73e856f2-29755920-n5h4b"]
Jul 29 20:00:14.200795 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.200757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:14.267072 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.267030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:14.335653 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.335590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:14.408313 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.408280    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"]
Jul 29 20:00:14.413978 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.413944    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-73e856f2-98hgl"]
Jul 29 20:00:14.420153 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.420105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:14.716226 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.716182    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63c97a0a-169f-436e-9335-20480f664261" path="/var/lib/kubelet/pods/63c97a0a-169f-436e-9335-20480f664261/volumes"
Jul 29 20:00:14.716588 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:14.716569    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6695e9ba-6b47-4329-b73e-4fa7a5317c42" path="/var/lib/kubelet/pods/6695e9ba-6b47-4329-b73e-4fa7a5317c42/volumes"
Jul 29 20:00:15.575725 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:15.575694    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:15.792797 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:15.792760    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:16.013433 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:16.013393    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:16.316169 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:16.316091    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:16.528125 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:16.528089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:18.100224 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:18.100191    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:00:18.680928 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:18.680895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:18.764739 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:18.764704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:18.833331 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:18.833300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:19.265918 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:19.265887    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:19.477120 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:19.477082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:19.573424 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:19.573329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:28.897506 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:28.897475    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:00:49.560153 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:49.560097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:49.626371 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:49.626340    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:50.135620 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:50.135579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:50.236039 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:50.235987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:50.636321 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:50.636282    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:50.738820 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:50.738785    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:50.836342 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:50.836305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:52.194232 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:52.194203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:53.202392 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:53.202350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:00:58.682809 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:58.682759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:58.793908 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:58.793866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:58.884767 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:58.884720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:59.241158 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:59.241121    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:00:59.409482 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:00:59.409444    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:25.366432 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:25.366389    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:31.630539 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:31.630496    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:31.738711 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:31.738680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:31.841868 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:31.841837    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:47.736826 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:47.736792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:50.384373 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:50.384337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:54.069704 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:54.069670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:54.172276 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:54.172233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:54.278524 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:54.278488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:01:59.565182 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:01:59.565141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:04.787244 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:04.787205    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:05.569315 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:05.569268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:12.180884 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:12.180840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:12.288604 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:12.288564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:12.381320 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:12.381278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:12.478171 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:12.478093    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:15.492407 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:15.492359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:02:23.942451 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:23.942398    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:24.034044 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:24.034013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:24.238287 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:24.238199    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:25.823023 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:25.822969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:27.803500 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:27.803469    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:02:31.786239 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:31.786160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:31.842697 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:31.842668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:32.544044 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:32.544009    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:37.796566 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:37.796530    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:02:38.567603 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:38.567560    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:38.627406 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:38.627369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:38.802965 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:38.802930    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:39.118744 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:39.118694    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:40.071310 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:40.071239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:41.941798 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:41.941757    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:42.039501 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:42.039448    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:42.134660 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:42.134623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:42.240059 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:42.239929    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:42.653369 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:42.653339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:42.755134 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:42.755098    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:02:47.597062 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:02:47.597022    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:03:06.998724 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:06.998685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:08.335617 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:08.335578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:13.864687 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:13.864655    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:13.982692 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:13.982654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:14.204646 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.204613    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"]
Jul 29 20:03:14.208052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.205522    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63c97a0a-169f-436e-9335-20480f664261" containerName="cleanup"
Jul 29 20:03:14.208052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.205575    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="63c97a0a-169f-436e-9335-20480f664261" containerName="cleanup"
Jul 29 20:03:14.208052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.205657    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6695e9ba-6b47-4329-b73e-4fa7a5317c42" containerName="revoke-keys"
Jul 29 20:03:14.208052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.205666    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6695e9ba-6b47-4329-b73e-4fa7a5317c42" containerName="revoke-keys"
Jul 29 20:03:14.208052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.205976    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6695e9ba-6b47-4329-b73e-4fa7a5317c42" containerName="revoke-keys"
Jul 29 20:03:14.208052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.206023    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="63c97a0a-169f-436e-9335-20480f664261" containerName="cleanup"
Jul 29 20:03:14.210364 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.210334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:14.213765 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.213392    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:03:14.216219 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.216194    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"]
Jul 29 20:03:14.347027 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.346959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6dad2f8-aa60-494d-ab93-0e6716abb2fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6\" (UID: \"d6dad2f8-aa60-494d-ab93-0e6716abb2fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:14.448369 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.448332    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6dad2f8-aa60-494d-ab93-0e6716abb2fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6\" (UID: \"d6dad2f8-aa60-494d-ab93-0e6716abb2fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:14.448987 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.448961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6dad2f8-aa60-494d-ab93-0e6716abb2fd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6\" (UID: \"d6dad2f8-aa60-494d-ab93-0e6716abb2fd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:14.524337 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.524256    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:14.667937 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:14.667886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"]
Jul 29 20:03:14.669259 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:03:14.669223    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6dad2f8_aa60_494d_ab93_0e6716abb2fd.slice/crio-fc611cedf48ccd142790aac7465f5bc6049e01aa24f2d810e358535329b38ffd WatchSource:0}: Error finding container fc611cedf48ccd142790aac7465f5bc6049e01aa24f2d810e358535329b38ffd: Status 404 returned error can't find the container with id fc611cedf48ccd142790aac7465f5bc6049e01aa24f2d810e358535329b38ffd
Jul 29 20:03:15.628581 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:15.628542    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6dad2f8-aa60-494d-ab93-0e6716abb2fd" containerID="b3ba260b72c340cd0ea7be1c93f418e79c944feb72e8a7f4dd5893f4f5b40540" exitCode=0
Jul 29 20:03:15.628981 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:15.628634    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6" event={"ID":"d6dad2f8-aa60-494d-ab93-0e6716abb2fd","Type":"ContainerDied","Data":"b3ba260b72c340cd0ea7be1c93f418e79c944feb72e8a7f4dd5893f4f5b40540"}
Jul 29 20:03:15.628981 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:15.628691    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6" event={"ID":"d6dad2f8-aa60-494d-ab93-0e6716abb2fd","Type":"ContainerStarted","Data":"fc611cedf48ccd142790aac7465f5bc6049e01aa24f2d810e358535329b38ffd"}
Jul 29 20:03:16.779711 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:16.779673    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:16.871862 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:16.871824    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6dad2f8-aa60-494d-ab93-0e6716abb2fd-maas-api-service-ca\") pod \"d6dad2f8-aa60-494d-ab93-0e6716abb2fd\" (UID: \"d6dad2f8-aa60-494d-ab93-0e6716abb2fd\") "
Jul 29 20:03:16.872204 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:16.872182    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6dad2f8-aa60-494d-ab93-0e6716abb2fd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "d6dad2f8-aa60-494d-ab93-0e6716abb2fd" (UID: "d6dad2f8-aa60-494d-ab93-0e6716abb2fd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:03:16.972608 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:16.972528    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6dad2f8-aa60-494d-ab93-0e6716abb2fd-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:03:17.640210 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:17.640178    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"
Jul 29 20:03:17.640381 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:17.640206    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6" event={"ID":"d6dad2f8-aa60-494d-ab93-0e6716abb2fd","Type":"ContainerDied","Data":"fc611cedf48ccd142790aac7465f5bc6049e01aa24f2d810e358535329b38ffd"}
Jul 29 20:03:17.640381 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:17.640254    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc611cedf48ccd142790aac7465f5bc6049e01aa24f2d810e358535329b38ffd"
Jul 29 20:03:24.851554 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:24.851507    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:24.951887 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:24.951854    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:25.051074 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:25.051034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:25.085696 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:25.085659    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"]
Jul 29 20:03:25.088828 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:25.088804    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-f42b0d30-ln6v6"]
Jul 29 20:03:25.110162 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:25.110079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:25.467853 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:25.467812    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:26.717120 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:26.717082    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6dad2f8-aa60-494d-ab93-0e6716abb2fd" path="/var/lib/kubelet/pods/d6dad2f8-aa60-494d-ab93-0e6716abb2fd/volumes"
Jul 29 20:03:27.502267 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:27.502229    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:03:31.522592 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:31.522554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:31.622620 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:31.622585    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:31.825086 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:31.824982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:32.040727 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:32.040693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:32.157489 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:32.157452    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:32.257971 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:32.257932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:32.424750 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:32.424660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:38.457294 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:38.457261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:38.518765 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:38.518737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:38.627168 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:38.627117    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:38.714441 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:38.714340    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:38.826269 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:38.826235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:45.109770 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:45.109713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:45.209631 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:45.209595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:51.957490 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:51.957451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:52.290166 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:52.290074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:52.941942 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:52.941900    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:55.312370 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:55.312329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:55.402479 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:55.402442    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:55.493253 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:55.493217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:55.598862 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:55.598781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:56.293395 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:56.293361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:03:56.389422 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:03:56.389392    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:04:01.910309 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:01.910225    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:04:25.371773 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:25.371733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:04:37.969290 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:37.969256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:04:38.777108 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:38.777082    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:04:38.780068 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:38.780041    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:04:38.783683 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:38.783660    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:04:38.786349 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:38.786331    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:04:40.906482 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:40.906443    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:04:55.300223 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:04:55.300188    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:05:13.899189 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:13.899153    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:05:15.427444 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:15.427404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:05:15.493470 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:15.493434    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:05:15.551803 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:15.551760    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:05:15.634512 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:15.634473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:05:18.761810 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:18.761779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:05:23.887461 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:23.887426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:05:24.399217 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:05:24.399173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:06:00.456673 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:00.456628    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:01.066725 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:01.066692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:06.727027 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:06.726953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:09.637231 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:09.637191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:09.691129 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:09.691079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:10.025563 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:10.025478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:30.355362 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:30.355327    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:30.455633 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:30.455595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:30.638850 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:30.638807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:30.868818 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:30.868780    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:31.904113 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:31.904082    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:32.003957 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:32.003924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:32.104189 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:32.104158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:32.768843 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:32.768808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:32.867288 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:32.867257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:32.966891 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:32.966858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:34.112760 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:34.112723    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:06:34.334866 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:34.334829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:38.864489 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:38.864456    2580 scope.go:117] "RemoveContainer" containerID="bf3d7718a77de5b647a9ba352ce3a4f7f9bf91f0e61f6abadce41df0c390c66b"
Jul 29 20:06:38.873778 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:38.873758    2580 scope.go:117] "RemoveContainer" containerID="dde17656ed96f2ea7fae631667fa0d6d69eb43b18893b1489abe5006268f0c9f"
Jul 29 20:06:40.524586 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:40.524546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:40.626659 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:40.626627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:42.473894 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:42.473857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:46.698897 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:46.698855    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:06:47.379183 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:47.379141    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:47.480885 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:47.480844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:47.581365 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:47.581336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:58.079055 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:58.079021    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:06:58.445583 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:06:58.445554    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:19.376067 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:19.376024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:26.351324 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:26.351286    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:26.454411 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:26.454377    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:26.697919 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:26.697882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:27.030626 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:27.030535    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:28.672713 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:28.672680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:30.318516 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:30.318486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:30.418184 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:30.418146    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:30.493456 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:30.493422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:30.593898 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:30.593809    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:31.022418 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:31.022386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:31.122244 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:31.122211    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:34.291049 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:34.291007    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:07:55.315200 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:55.315160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:55.656178 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:55.656143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:55.980522 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:55.980418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:56.713592 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:56.713546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:58.464195 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:58.464154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:58.549383 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:58.549349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:58.612987 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:58.612953    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:59.285801 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:59.285745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:59.380111 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:59.380065    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:07:59.657389 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:07:59.657352    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:08:05.993367 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:08:05.993330    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:08:23.469430 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:08:23.469398    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:08:37.377210 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:08:37.377118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:08:40.993358 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:08:40.993316    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:08:51.592963 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:08:51.592922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:09:14.706533 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:14.706469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:09:17.796089 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:17.796055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:09:31.005751 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:31.005658    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:09:38.811594 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:38.811565    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:09:38.815679 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:38.815657    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:09:38.818323 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:38.818304    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:09:38.822065 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:38.822047    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:09:38.919057 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:38.919024    2580 scope.go:117] "RemoveContainer" containerID="b3ba260b72c340cd0ea7be1c93f418e79c944feb72e8a7f4dd5893f4f5b40540"
Jul 29 20:09:58.360262 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:58.360229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:09:59.306681 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.306636    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:09:59.383119 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.383075    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"]
Jul 29 20:09:59.383704 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.383684    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6dad2f8-aa60-494d-ab93-0e6716abb2fd" containerName="revoke-keys"
Jul 29 20:09:59.383794 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.383708    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6dad2f8-aa60-494d-ab93-0e6716abb2fd" containerName="revoke-keys"
Jul 29 20:09:59.383848 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.383822    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6dad2f8-aa60-494d-ab93-0e6716abb2fd" containerName="revoke-keys"
Jul 29 20:09:59.387076 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.387049    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:09:59.388684 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.388660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:09:59.395513 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.395070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"]
Jul 29 20:09:59.508897 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.508855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66c59c9d-84e7-42a4-a52d-2b5ce31c395c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g\" (UID: \"66c59c9d-84e7-42a4-a52d-2b5ce31c395c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:09:59.609754 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.609659    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66c59c9d-84e7-42a4-a52d-2b5ce31c395c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g\" (UID: \"66c59c9d-84e7-42a4-a52d-2b5ce31c395c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:09:59.610269 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.610252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66c59c9d-84e7-42a4-a52d-2b5ce31c395c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g\" (UID: \"66c59c9d-84e7-42a4-a52d-2b5ce31c395c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:09:59.700576 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.700500    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:09:59.842036 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.841867    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"]
Jul 29 20:09:59.845688 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:09:59.845653    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66c59c9d_84e7_42a4_a52d_2b5ce31c395c.slice/crio-97c7865aafc93d9ce4e28e5a1a0a6d1ed5d3b8e46fc18a806d4dd3e58c1d86a7 WatchSource:0}: Error finding container 97c7865aafc93d9ce4e28e5a1a0a6d1ed5d3b8e46fc18a806d4dd3e58c1d86a7: Status 404 returned error can't find the container with id 97c7865aafc93d9ce4e28e5a1a0a6d1ed5d3b8e46fc18a806d4dd3e58c1d86a7
Jul 29 20:09:59.891895 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:09:59.891853    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g" event={"ID":"66c59c9d-84e7-42a4-a52d-2b5ce31c395c","Type":"ContainerStarted","Data":"97c7865aafc93d9ce4e28e5a1a0a6d1ed5d3b8e46fc18a806d4dd3e58c1d86a7"}
Jul 29 20:10:00.899495 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:00.899462    2580 generic.go:356] "Generic (PLEG): container finished" podID="66c59c9d-84e7-42a4-a52d-2b5ce31c395c" containerID="8073c2a9a737ffbbb9682b4972bd73eeb0728b3e7fb9eb8e1305e4a4cde100c3" exitCode=0
Jul 29 20:10:00.899870 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:00.899503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g" event={"ID":"66c59c9d-84e7-42a4-a52d-2b5ce31c395c","Type":"ContainerDied","Data":"8073c2a9a737ffbbb9682b4972bd73eeb0728b3e7fb9eb8e1305e4a4cde100c3"}
Jul 29 20:10:02.048895 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.048863    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:10:02.234378 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.234282    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66c59c9d-84e7-42a4-a52d-2b5ce31c395c-maas-api-service-ca\") pod \"66c59c9d-84e7-42a4-a52d-2b5ce31c395c\" (UID: \"66c59c9d-84e7-42a4-a52d-2b5ce31c395c\") "
Jul 29 20:10:02.234634 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.234611    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66c59c9d-84e7-42a4-a52d-2b5ce31c395c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66c59c9d-84e7-42a4-a52d-2b5ce31c395c" (UID: "66c59c9d-84e7-42a4-a52d-2b5ce31c395c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:10:02.336119 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.336088    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66c59c9d-84e7-42a4-a52d-2b5ce31c395c-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:10:02.796804 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.796765    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:10:02.918307 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.918276    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"
Jul 29 20:10:02.918480 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.918313    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g" event={"ID":"66c59c9d-84e7-42a4-a52d-2b5ce31c395c","Type":"ContainerDied","Data":"97c7865aafc93d9ce4e28e5a1a0a6d1ed5d3b8e46fc18a806d4dd3e58c1d86a7"}
Jul 29 20:10:02.918480 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:02.918379    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97c7865aafc93d9ce4e28e5a1a0a6d1ed5d3b8e46fc18a806d4dd3e58c1d86a7"
Jul 29 20:10:10.018783 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.018741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:10.099365 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.099336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:10.224101 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.224071    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:10.239060 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.239028    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"]
Jul 29 20:10:10.245075 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.245048    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f5ad4c-vm85g"]
Jul 29 20:10:10.278464 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.278385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:10.717300 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.717261    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66c59c9d-84e7-42a4-a52d-2b5ce31c395c" path="/var/lib/kubelet/pods/66c59c9d-84e7-42a4-a52d-2b5ce31c395c/volumes"
Jul 29 20:10:10.891399 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:10.891361    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:15.604406 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:15.604374    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:10:16.982875 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:16.982842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:17.044323 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.044282    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:17.269284 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.269192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:17.485366 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.485330    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:17.601599 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.601515    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"]
Jul 29 20:10:17.602092 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.602067    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66c59c9d-84e7-42a4-a52d-2b5ce31c395c" containerName="revoke-keys"
Jul 29 20:10:17.602092 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.602087    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="66c59c9d-84e7-42a4-a52d-2b5ce31c395c" containerName="revoke-keys"
Jul 29 20:10:17.602266 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.602166    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="66c59c9d-84e7-42a4-a52d-2b5ce31c395c" containerName="revoke-keys"
Jul 29 20:10:17.605285 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.605268    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:17.607053 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.607034    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:10:17.611951 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.611915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"]
Jul 29 20:10:17.683421 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.683379    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4978ebbf-a834-4388-9094-f9e544484846-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-620565-6vqqr\" (UID: \"4978ebbf-a834-4388-9094-f9e544484846\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:17.784561 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.784523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4978ebbf-a834-4388-9094-f9e544484846-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-620565-6vqqr\" (UID: \"4978ebbf-a834-4388-9094-f9e544484846\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:17.785252 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.785230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4978ebbf-a834-4388-9094-f9e544484846-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-620565-6vqqr\" (UID: \"4978ebbf-a834-4388-9094-f9e544484846\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:17.917486 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.917452    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:17.937712 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:17.937684    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:18.062700 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:18.062676    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"]
Jul 29 20:10:18.064505 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:10:18.064476    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4978ebbf_a834_4388_9094_f9e544484846.slice/crio-33d40d2ac3e25165600ac2a80c2e411e9581914e7ad295bccf23a2c4cb20eb53 WatchSource:0}: Error finding container 33d40d2ac3e25165600ac2a80c2e411e9581914e7ad295bccf23a2c4cb20eb53: Status 404 returned error can't find the container with id 33d40d2ac3e25165600ac2a80c2e411e9581914e7ad295bccf23a2c4cb20eb53
Jul 29 20:10:19.013489 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:19.013452    2580 generic.go:356] "Generic (PLEG): container finished" podID="4978ebbf-a834-4388-9094-f9e544484846" containerID="40c1f2cf1a60480bf77bb039f4dfd0d5ed0f6bd52cc3ea3a0b4ef86777d08ac0" exitCode=0
Jul 29 20:10:19.013639 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:19.013546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr" event={"ID":"4978ebbf-a834-4388-9094-f9e544484846","Type":"ContainerDied","Data":"40c1f2cf1a60480bf77bb039f4dfd0d5ed0f6bd52cc3ea3a0b4ef86777d08ac0"}
Jul 29 20:10:19.013639 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:19.013600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr" event={"ID":"4978ebbf-a834-4388-9094-f9e544484846","Type":"ContainerStarted","Data":"33d40d2ac3e25165600ac2a80c2e411e9581914e7ad295bccf23a2c4cb20eb53"}
Jul 29 20:10:20.164489 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:20.164462    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:20.209185 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:20.209144    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4978ebbf-a834-4388-9094-f9e544484846-maas-api-service-ca\") pod \"4978ebbf-a834-4388-9094-f9e544484846\" (UID: \"4978ebbf-a834-4388-9094-f9e544484846\") "
Jul 29 20:10:20.209500 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:20.209478    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4978ebbf-a834-4388-9094-f9e544484846-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4978ebbf-a834-4388-9094-f9e544484846" (UID: "4978ebbf-a834-4388-9094-f9e544484846"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:10:20.310364 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:20.310271    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4978ebbf-a834-4388-9094-f9e544484846-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:10:21.025295 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:21.025218    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"
Jul 29 20:10:21.025295 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:21.025242    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr" event={"ID":"4978ebbf-a834-4388-9094-f9e544484846","Type":"ContainerDied","Data":"33d40d2ac3e25165600ac2a80c2e411e9581914e7ad295bccf23a2c4cb20eb53"}
Jul 29 20:10:21.025295 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:21.025288    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="33d40d2ac3e25165600ac2a80c2e411e9581914e7ad295bccf23a2c4cb20eb53"
Jul 29 20:10:28.214269 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.214231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:28.314708 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.314675    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:28.414433 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.414403    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:28.433547 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.433519    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"]
Jul 29 20:10:28.436306 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.436282    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-620565-6vqqr"]
Jul 29 20:10:28.632038 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.631990    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:28.705169 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.705135    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:28.717600 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.717568    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4978ebbf-a834-4388-9094-f9e544484846" path="/var/lib/kubelet/pods/4978ebbf-a834-4388-9094-f9e544484846/volumes"
Jul 29 20:10:28.972676 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:28.972598    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:31.299398 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:31.299361    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:10:35.203164 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:35.203127    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:35.263684 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:35.263650    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:35.490457 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:35.490339    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:35.696892 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:35.696857    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:36.044254 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:36.044217    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:36.351451 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:36.351371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:36.805346 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:36.805313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:38.584445 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:38.584405    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:39.348841 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:39.348804    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:39.446804 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:39.446774    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:39.508455 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:39.508422    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:40.120657 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:40.120619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:40.221819 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:40.221783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:40.310928 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:40.310898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:10:45.598474 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:45.598435    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:10:48.688092 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:10:48.688058    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:04.239676 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:04.239626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:04.838916 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:04.838883    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:05.120164 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:05.120075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:05.260948 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:05.260904    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:08.417778 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:08.417741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:08.509634 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:08.509603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:08.579050 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:08.578990    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:09.204076 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:09.204043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:09.304168 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:09.304132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:09.377432 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:09.377397    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:14.407327 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:14.407124    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:11:33.376985 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:33.376945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:36.223111 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.223064    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"]
Jul 29 20:11:36.223674 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.223504    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4978ebbf-a834-4388-9094-f9e544484846" containerName="revoke-keys"
Jul 29 20:11:36.223674 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.223516    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4978ebbf-a834-4388-9094-f9e544484846" containerName="revoke-keys"
Jul 29 20:11:36.223674 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.223593    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4978ebbf-a834-4388-9094-f9e544484846" containerName="revoke-keys"
Jul 29 20:11:36.226831 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.226804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:36.228689 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.228666    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:11:36.233489 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.233465    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"]
Jul 29 20:11:36.347733 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.347685    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-78b35e-hxcks\" (UID: \"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:36.448341 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.448301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-78b35e-hxcks\" (UID: \"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:36.449027 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.448968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-78b35e-hxcks\" (UID: \"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:36.539373 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.539294    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:36.678766 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:36.678740    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"]
Jul 29 20:11:36.681695 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:11:36.681667    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfab0a3da_0b18_4a2a_8f31_599dd70f7cc2.slice/crio-3b4886ca22a79c0466d1bdb21bc674ae525f879c807aef1cda0204edb804357c WatchSource:0}: Error finding container 3b4886ca22a79c0466d1bdb21bc674ae525f879c807aef1cda0204edb804357c: Status 404 returned error can't find the container with id 3b4886ca22a79c0466d1bdb21bc674ae525f879c807aef1cda0204edb804357c
Jul 29 20:11:37.451724 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:37.451694    2580 generic.go:356] "Generic (PLEG): container finished" podID="fab0a3da-0b18-4a2a-8f31-599dd70f7cc2" containerID="7968d9234a3836be870f0fd83cd912c3afe2b348b822cb9d001475aba96cd4e4" exitCode=0
Jul 29 20:11:37.452084 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:37.451786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks" event={"ID":"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2","Type":"ContainerDied","Data":"7968d9234a3836be870f0fd83cd912c3afe2b348b822cb9d001475aba96cd4e4"}
Jul 29 20:11:37.452084 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:37.451847    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks" event={"ID":"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2","Type":"ContainerStarted","Data":"3b4886ca22a79c0466d1bdb21bc674ae525f879c807aef1cda0204edb804357c"}
Jul 29 20:11:38.611890 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:38.611856    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:38.671954 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:38.671920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2-maas-api-service-ca\") pod \"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2\" (UID: \"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2\") "
Jul 29 20:11:38.672361 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:38.672337    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fab0a3da-0b18-4a2a-8f31-599dd70f7cc2" (UID: "fab0a3da-0b18-4a2a-8f31-599dd70f7cc2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:11:38.773886 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:38.773849    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:11:39.463534 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:39.463508    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"
Jul 29 20:11:39.463534 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:39.463526    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks" event={"ID":"fab0a3da-0b18-4a2a-8f31-599dd70f7cc2","Type":"ContainerDied","Data":"3b4886ca22a79c0466d1bdb21bc674ae525f879c807aef1cda0204edb804357c"}
Jul 29 20:11:39.463769 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:39.463573    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b4886ca22a79c0466d1bdb21bc674ae525f879c807aef1cda0204edb804357c"
Jul 29 20:11:45.902017 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:45.901927    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:46.001905 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.001872    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:46.103549 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.103515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:46.105036 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.104988    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"]
Jul 29 20:11:46.108359 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.108337    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-78b35e-hxcks"]
Jul 29 20:11:46.202362 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.202276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:46.468764 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.468686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:46.716950 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:46.716908    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fab0a3da-0b18-4a2a-8f31-599dd70f7cc2" path="/var/lib/kubelet/pods/fab0a3da-0b18-4a2a-8f31-599dd70f7cc2/volumes"
Jul 29 20:11:49.002461 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:49.002425    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:11:52.589271 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:52.589229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:52.689571 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:52.689536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:52.865705 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:52.865590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:11:53.081567 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:11:53.081532    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:08.791167 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:08.791131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:08.860985 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:08.860950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:08.925112 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:08.925076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:08.994657 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:08.994627    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:09.352222 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:09.352181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:13.801419 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:13.801384    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:12:15.492946 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:15.492905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:15.562915 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:15.562878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:15.754836 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:15.754753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:16.047756 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:16.047656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:16.315304 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:16.315218    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:16.614740 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:16.614639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:17.142500 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:17.142464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:19.795008 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:19.794958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:19.873424 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:19.873384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:19.949043 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:19.948988    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:20.399940 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:20.399903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:20.510080 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:20.510040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:20.610631 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:20.610595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:24.997433 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:24.997394    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:12:44.753058 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:44.752971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:45.082939 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:45.082823    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:45.378248 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:45.378213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:45.618469 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:45.618435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:47.935014 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:47.934969    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:48.020765 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:48.020726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:48.086531 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:48.086493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:48.188307 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:48.188203    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:48.678834 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:48.678789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:48.778521 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:48.778480    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:12:55.496182 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:12:55.496146    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:13:12.929686 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:13:12.929647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:13:17.906491 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:13:17.906459    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:13:21.297965 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:13:21.297925    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:13:44.615284 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:13:44.615241    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:13:48.292488 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:13:48.292448    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:14:19.395460 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:19.395424    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:14:38.856335 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:38.856258    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:14:38.861896 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:38.861863    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:14:38.863142 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:38.863120    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:14:38.868216 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:38.868199    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:14:39.799825 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:39.799785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:14:44.167545 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:44.167513    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:14:44.222459 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:44.222419    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:14:53.002688 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:14:53.002651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:15:00.133071 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.133026    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"]
Jul 29 20:15:00.133676 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.133654    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fab0a3da-0b18-4a2a-8f31-599dd70f7cc2" containerName="revoke-keys"
Jul 29 20:15:00.133762 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.133679    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fab0a3da-0b18-4a2a-8f31-599dd70f7cc2" containerName="revoke-keys"
Jul 29 20:15:00.133840 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.133828    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fab0a3da-0b18-4a2a-8f31-599dd70f7cc2" containerName="revoke-keys"
Jul 29 20:15:00.136753 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.136731    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"]
Jul 29 20:15:00.136884 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.136847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"
Jul 29 20:15:00.138800 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.138779    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:15:00.140615 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.140597    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"]
Jul 29 20:15:00.140742 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.140707    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"
Jul 29 20:15:00.144339 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.144319    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"
Jul 29 20:15:00.146245 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.146123    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"]
Jul 29 20:15:00.148247 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.148222    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"]
Jul 29 20:15:00.150960 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.150936    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"]
Jul 29 20:15:00.153051 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.152796    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"
Jul 29 20:15:00.160878 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.160861    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"
Jul 29 20:15:00.197316 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.197289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"
Jul 29 20:15:00.322963 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.322935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"]
Jul 29 20:15:00.325371 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:15:00.325335    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod707504db_4880_42c3_8198_7a7d26dda73e.slice/crio-272a29c209c1e82d022f48042f814bf292d8c5c121cf1f19f74742ffcc4895a4 WatchSource:0}: Error finding container 272a29c209c1e82d022f48042f814bf292d8c5c121cf1f19f74742ffcc4895a4: Status 404 returned error can't find the container with id 272a29c209c1e82d022f48042f814bf292d8c5c121cf1f19f74742ffcc4895a4
Jul 29 20:15:00.390604 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.390580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"]
Jul 29 20:15:00.392445 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:15:00.392415    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c8fa379_840b_4445_b8d1_68983ea62be8.slice/crio-b59971bfe90a157088ac1bfa3bccb94665307ad9fe50390dbe7f9b2be78c08d0 WatchSource:0}: Error finding container b59971bfe90a157088ac1bfa3bccb94665307ad9fe50390dbe7f9b2be78c08d0: Status 404 returned error can't find the container with id b59971bfe90a157088ac1bfa3bccb94665307ad9fe50390dbe7f9b2be78c08d0
Jul 29 20:15:00.560850 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.560823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"]
Jul 29 20:15:00.562773 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:15:00.562734    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a440564_314c_45f9_9fb3_28c6f7e1dc61.slice/crio-c8b09c8109405c54a72932411a97190ae94cd9cb30c579226246fd54d55adb3a WatchSource:0}: Error finding container c8b09c8109405c54a72932411a97190ae94cd9cb30c579226246fd54d55adb3a: Status 404 returned error can't find the container with id c8b09c8109405c54a72932411a97190ae94cd9cb30c579226246fd54d55adb3a
Jul 29 20:15:00.613828 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.613795    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx" event={"ID":"6c8fa379-840b-4445-b8d1-68983ea62be8","Type":"ContainerStarted","Data":"b59971bfe90a157088ac1bfa3bccb94665307ad9fe50390dbe7f9b2be78c08d0"}
Jul 29 20:15:00.615415 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.615384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp" event={"ID":"707504db-4880-42c3-8198-7a7d26dda73e","Type":"ContainerStarted","Data":"272a29c209c1e82d022f48042f814bf292d8c5c121cf1f19f74742ffcc4895a4"}
Jul 29 20:15:00.617012 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:00.616952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62" event={"ID":"1a440564-314c-45f9-9fb3-28c6f7e1dc61","Type":"ContainerStarted","Data":"c8b09c8109405c54a72932411a97190ae94cd9cb30c579226246fd54d55adb3a"}
Jul 29 20:15:01.625195 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:01.625164    2580 generic.go:356] "Generic (PLEG): container finished" podID="707504db-4880-42c3-8198-7a7d26dda73e" containerID="d61f4bb8dd323fe7433e5c058f3aade3b262f531c56f3d3597b0b1ea373caa82" exitCode=0
Jul 29 20:15:01.625572 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:01.625239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp" event={"ID":"707504db-4880-42c3-8198-7a7d26dda73e","Type":"ContainerDied","Data":"d61f4bb8dd323fe7433e5c058f3aade3b262f531c56f3d3597b0b1ea373caa82"}
Jul 29 20:15:01.626866 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:01.626841    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62" event={"ID":"1a440564-314c-45f9-9fb3-28c6f7e1dc61","Type":"ContainerStarted","Data":"13e018e66653706bd501d4778cac34ce2b2b9d744523690f6f7febdf6f9ec30e"}
Jul 29 20:15:01.628225 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:01.628207    2580 generic.go:356] "Generic (PLEG): container finished" podID="6c8fa379-840b-4445-b8d1-68983ea62be8" containerID="745694706eb463cf2b598ab1f4444e4d027a7b68fa10db1a98690ad79684e2a2" exitCode=0
Jul 29 20:15:01.628299 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:01.628234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx" event={"ID":"6c8fa379-840b-4445-b8d1-68983ea62be8","Type":"ContainerDied","Data":"745694706eb463cf2b598ab1f4444e4d027a7b68fa10db1a98690ad79684e2a2"}
Jul 29 20:15:02.635828 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:02.635796    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a440564-314c-45f9-9fb3-28c6f7e1dc61" containerID="13e018e66653706bd501d4778cac34ce2b2b9d744523690f6f7febdf6f9ec30e" exitCode=0
Jul 29 20:15:02.636321 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:02.635889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62" event={"ID":"1a440564-314c-45f9-9fb3-28c6f7e1dc61","Type":"ContainerDied","Data":"13e018e66653706bd501d4778cac34ce2b2b9d744523690f6f7febdf6f9ec30e"}
Jul 29 20:15:02.803197 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:02.803168    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"
Jul 29 20:15:02.807714 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:02.807685    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"
Jul 29 20:15:02.842013 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:02.841972    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"
Jul 29 20:15:03.642967 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.642939    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"
Jul 29 20:15:03.642967 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.642951    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp" event={"ID":"707504db-4880-42c3-8198-7a7d26dda73e","Type":"ContainerDied","Data":"272a29c209c1e82d022f48042f814bf292d8c5c121cf1f19f74742ffcc4895a4"}
Jul 29 20:15:03.643473 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.643024    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="272a29c209c1e82d022f48042f814bf292d8c5c121cf1f19f74742ffcc4895a4"
Jul 29 20:15:03.644229 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.644208    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62"
Jul 29 20:15:03.644340 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.644251    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29755935-7zl62" event={"ID":"1a440564-314c-45f9-9fb3-28c6f7e1dc61","Type":"ContainerDied","Data":"c8b09c8109405c54a72932411a97190ae94cd9cb30c579226246fd54d55adb3a"}
Jul 29 20:15:03.644340 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.644290    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c8b09c8109405c54a72932411a97190ae94cd9cb30c579226246fd54d55adb3a"
Jul 29 20:15:03.645499 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.645477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx" event={"ID":"6c8fa379-840b-4445-b8d1-68983ea62be8","Type":"ContainerDied","Data":"b59971bfe90a157088ac1bfa3bccb94665307ad9fe50390dbe7f9b2be78c08d0"}
Jul 29 20:15:03.645618 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.645515    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"
Jul 29 20:15:03.645618 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:03.645519    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b59971bfe90a157088ac1bfa3bccb94665307ad9fe50390dbe7f9b2be78c08d0"
Jul 29 20:15:04.098473 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:04.098382    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:15:18.894783 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:18.894743    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:15:37.907249 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:15:37.907209    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:16:03.789400 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:03.789314    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:16:04.087071 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:04.086955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:16:05.702814 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:05.702774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:16:19.292362 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:19.292325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:16:31.316598 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:31.316554    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:16:38.992064 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:38.992031    2580 scope.go:117] "RemoveContainer" containerID="40c1f2cf1a60480bf77bb039f4dfd0d5ed0f6bd52cc3ea3a0b4ef86777d08ac0"
Jul 29 20:16:39.001648 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:39.001627    2580 scope.go:117] "RemoveContainer" containerID="8073c2a9a737ffbbb9682b4972bd73eeb0728b3e7fb9eb8e1305e4a4cde100c3"
Jul 29 20:16:49.401349 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:16:49.401313    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:17:07.790768 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:07.790735    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:17:34.990565 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:34.990479    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:17:39.032726 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:39.032692    2580 scope.go:117] "RemoveContainer" containerID="7968d9234a3836be870f0fd83cd912c3afe2b348b822cb9d001475aba96cd4e4"
Jul 29 20:17:46.730579 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:46.730538    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:17:46.832234 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:46.832196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:17:48.299340 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:48.299300    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:17:58.843048 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:17:58.843014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:18:01.098471 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:01.098428    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:18:18.003479 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:18.003440    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:18:21.191880 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:21.191840    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:18:22.216717 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:22.216684    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:18:26.003531 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:26.003499    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:18:34.893275 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:34.893237    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:18:42.070140 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:42.070080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:18:47.188187 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:47.188154    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:18:50.697032 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:50.696971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:18:54.674866 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:18:54.674830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:19:20.404759 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:20.404719    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:19:38.899314 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:38.899270    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:19:38.906836 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:38.906809    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:19:38.907887 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:38.907858    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:19:38.914815 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:38.914792    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:19:39.191509 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:39.191415    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:19:45.306115 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:45.306079    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:19:46.193295 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:46.193263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:19:55.801031 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:55.800981    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:19:59.328946 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:19:59.328913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:20:03.998677 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:20:03.998643    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:20:20.903789 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:20:20.903749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:20:24.289938 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:20:24.289898    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:20:54.404712 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:20:54.404674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:21:13.496955 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:13.496918    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:21:19.027055 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:19.026979    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:19.932493 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:19.932451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:20.802686 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.802650    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7500-140135897996336"]
Jul 29 20:21:20.803108 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803093    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a440564-314c-45f9-9fb3-28c6f7e1dc61" containerName="cleanup"
Jul 29 20:21:20.803108 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803108    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a440564-314c-45f9-9fb3-28c6f7e1dc61" containerName="cleanup"
Jul 29 20:21:20.803186 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803121    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c8fa379-840b-4445-b8d1-68983ea62be8" containerName="cleanup"
Jul 29 20:21:20.803186 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803127    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c8fa379-840b-4445-b8d1-68983ea62be8" containerName="cleanup"
Jul 29 20:21:20.803186 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803139    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="707504db-4880-42c3-8198-7a7d26dda73e" containerName="cleanup"
Jul 29 20:21:20.803186 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803144    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="707504db-4880-42c3-8198-7a7d26dda73e" containerName="cleanup"
Jul 29 20:21:20.803310 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803217    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a440564-314c-45f9-9fb3-28c6f7e1dc61" containerName="cleanup"
Jul 29 20:21:20.803310 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803228    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c8fa379-840b-4445-b8d1-68983ea62be8" containerName="cleanup"
Jul 29 20:21:20.803310 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.803238    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="707504db-4880-42c3-8198-7a7d26dda73e" containerName="cleanup"
Jul 29 20:21:20.806473 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.806457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:20.808234 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.808211    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mmdfg\""
Jul 29 20:21:20.812263 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.812239    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897996336"]
Jul 29 20:21:20.911264 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:20.911216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw247\" (UniqueName: \"kubernetes.io/projected/7e08aa2e-376c-4389-b4b3-827a7b91b92a-kube-api-access-sw247\") pod \"test-curl-7500-140135897996336\" (UID: \"7e08aa2e-376c-4389-b4b3-827a7b91b92a\") " pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:21.012824 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:21.012783    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw247\" (UniqueName: \"kubernetes.io/projected/7e08aa2e-376c-4389-b4b3-827a7b91b92a-kube-api-access-sw247\") pod \"test-curl-7500-140135897996336\" (UID: \"7e08aa2e-376c-4389-b4b3-827a7b91b92a\") " pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:21.021820 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:21.021789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw247\" (UniqueName: \"kubernetes.io/projected/7e08aa2e-376c-4389-b4b3-827a7b91b92a-kube-api-access-sw247\") pod \"test-curl-7500-140135897996336\" (UID: \"7e08aa2e-376c-4389-b4b3-827a7b91b92a\") " pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:21.117601 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:21.117502    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:21.259844 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:21.259823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897996336"]
Jul 29 20:21:21.262230 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:21:21.262199    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e08aa2e_376c_4389_b4b3_827a7b91b92a.slice/crio-2f4a1003fe67e2fc358bb9c9918ae456515a475ca8bf746a7cf930c68ad0fec7 WatchSource:0}: Error finding container 2f4a1003fe67e2fc358bb9c9918ae456515a475ca8bf746a7cf930c68ad0fec7: Status 404 returned error can't find the container with id 2f4a1003fe67e2fc358bb9c9918ae456515a475ca8bf746a7cf930c68ad0fec7
Jul 29 20:21:21.263631 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:21.263607    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:21:21.786043 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:21.786009    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897996336" event={"ID":"7e08aa2e-376c-4389-b4b3-827a7b91b92a","Type":"ContainerStarted","Data":"2f4a1003fe67e2fc358bb9c9918ae456515a475ca8bf746a7cf930c68ad0fec7"}
Jul 29 20:21:22.792502 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:22.792460    2580 generic.go:356] "Generic (PLEG): container finished" podID="7e08aa2e-376c-4389-b4b3-827a7b91b92a" containerID="7e5d63c8277ecdd505f94e7199f7cf40a598bacd2aa2f212016380d9e7ec07bb" exitCode=0
Jul 29 20:21:22.792907 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:22.792556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897996336" event={"ID":"7e08aa2e-376c-4389-b4b3-827a7b91b92a","Type":"ContainerDied","Data":"7e5d63c8277ecdd505f94e7199f7cf40a598bacd2aa2f212016380d9e7ec07bb"}
Jul 29 20:21:23.951803 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:23.951777    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:23.974212 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:23.974166    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7500-140135897996336_7e08aa2e-376c-4389-b4b3-827a7b91b92a/test-curl-7500-140135897996336/0.log"
Jul 29 20:21:23.995825 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:23.995791    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7500-140135897996336"]
Jul 29 20:21:24.000803 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.000777    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7500-140135897996336"]
Jul 29 20:21:24.042843 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.042814    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sw247\" (UniqueName: \"kubernetes.io/projected/7e08aa2e-376c-4389-b4b3-827a7b91b92a-kube-api-access-sw247\") pod \"7e08aa2e-376c-4389-b4b3-827a7b91b92a\" (UID: \"7e08aa2e-376c-4389-b4b3-827a7b91b92a\") "
Jul 29 20:21:24.045254 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.045232    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e08aa2e-376c-4389-b4b3-827a7b91b92a-kube-api-access-sw247" (OuterVolumeSpecName: "kube-api-access-sw247") pod "7e08aa2e-376c-4389-b4b3-827a7b91b92a" (UID: "7e08aa2e-376c-4389-b4b3-827a7b91b92a"). InnerVolumeSpecName "kube-api-access-sw247". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:21:24.114518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.114473    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7500-140135897994544"]
Jul 29 20:21:24.114925 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.114911    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e08aa2e-376c-4389-b4b3-827a7b91b92a" containerName="test-curl-7500-140135897996336"
Jul 29 20:21:24.114973 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.114926    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e08aa2e-376c-4389-b4b3-827a7b91b92a" containerName="test-curl-7500-140135897996336"
Jul 29 20:21:24.115049 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.115038    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e08aa2e-376c-4389-b4b3-827a7b91b92a" containerName="test-curl-7500-140135897996336"
Jul 29 20:21:24.118283 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.118265    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:24.124953 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.124923    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897994544"]
Jul 29 20:21:24.144425 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.144386    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sw247\" (UniqueName: \"kubernetes.io/projected/7e08aa2e-376c-4389-b4b3-827a7b91b92a-kube-api-access-sw247\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:21:24.245120 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.245079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sqw7\" (UniqueName: \"kubernetes.io/projected/4f072d64-7b8b-4bcd-a769-8a7026a448e6-kube-api-access-5sqw7\") pod \"test-curl-7500-140135897994544\" (UID: \"4f072d64-7b8b-4bcd-a769-8a7026a448e6\") " pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:24.346050 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.345916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sqw7\" (UniqueName: \"kubernetes.io/projected/4f072d64-7b8b-4bcd-a769-8a7026a448e6-kube-api-access-5sqw7\") pod \"test-curl-7500-140135897994544\" (UID: \"4f072d64-7b8b-4bcd-a769-8a7026a448e6\") " pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:24.354435 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.354406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sqw7\" (UniqueName: \"kubernetes.io/projected/4f072d64-7b8b-4bcd-a769-8a7026a448e6-kube-api-access-5sqw7\") pod \"test-curl-7500-140135897994544\" (UID: \"4f072d64-7b8b-4bcd-a769-8a7026a448e6\") " pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:24.428728 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.428691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:24.568192 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.568161    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897994544"]
Jul 29 20:21:24.568826 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:21:24.568800    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f072d64_7b8b_4bcd_a769_8a7026a448e6.slice/crio-0ae9aa369e2627393dda83e923f67a305482e61abbf0cf61e6bf7fe14fa24419 WatchSource:0}: Error finding container 0ae9aa369e2627393dda83e923f67a305482e61abbf0cf61e6bf7fe14fa24419: Status 404 returned error can't find the container with id 0ae9aa369e2627393dda83e923f67a305482e61abbf0cf61e6bf7fe14fa24419
Jul 29 20:21:24.716103 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.716069    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e08aa2e-376c-4389-b4b3-827a7b91b92a" path="/var/lib/kubelet/pods/7e08aa2e-376c-4389-b4b3-827a7b91b92a/volumes"
Jul 29 20:21:24.804195 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.804161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897994544" event={"ID":"4f072d64-7b8b-4bcd-a769-8a7026a448e6","Type":"ContainerStarted","Data":"0ae9aa369e2627393dda83e923f67a305482e61abbf0cf61e6bf7fe14fa24419"}
Jul 29 20:21:24.805312 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.805291    2580 scope.go:117] "RemoveContainer" containerID="7e5d63c8277ecdd505f94e7199f7cf40a598bacd2aa2f212016380d9e7ec07bb"
Jul 29 20:21:24.805435 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:24.805321    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897996336"
Jul 29 20:21:25.814777 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:25.814737    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" containerID="91cea8091e0b136c27b2aa6c60813770eecd4deddb2ff5e868c0550b7da1f16e" exitCode=0
Jul 29 20:21:25.814777 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:25.814779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897994544" event={"ID":"4f072d64-7b8b-4bcd-a769-8a7026a448e6","Type":"ContainerDied","Data":"91cea8091e0b136c27b2aa6c60813770eecd4deddb2ff5e868c0550b7da1f16e"}
Jul 29 20:21:26.966578 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:26.966547    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:26.986780 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:26.986750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7500-140135897994544_4f072d64-7b8b-4bcd-a769-8a7026a448e6/test-curl-7500-140135897994544/0.log"
Jul 29 20:21:27.009785 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.009748    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7500-140135897994544"]
Jul 29 20:21:27.015529 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.015495    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7500-140135897994544"]
Jul 29 20:21:27.080510 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.080402    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sqw7\" (UniqueName: \"kubernetes.io/projected/4f072d64-7b8b-4bcd-a769-8a7026a448e6-kube-api-access-5sqw7\") pod \"4f072d64-7b8b-4bcd-a769-8a7026a448e6\" (UID: \"4f072d64-7b8b-4bcd-a769-8a7026a448e6\") "
Jul 29 20:21:27.082753 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.082725    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f072d64-7b8b-4bcd-a769-8a7026a448e6-kube-api-access-5sqw7" (OuterVolumeSpecName: "kube-api-access-5sqw7") pod "4f072d64-7b8b-4bcd-a769-8a7026a448e6" (UID: "4f072d64-7b8b-4bcd-a769-8a7026a448e6"). InnerVolumeSpecName "kube-api-access-5sqw7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:21:27.173687 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.173644    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:27.174214 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.174195    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" containerName="test-curl-7500-140135897994544"
Jul 29 20:21:27.174326 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.174217    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" containerName="test-curl-7500-140135897994544"
Jul 29 20:21:27.174382 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.174357    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" containerName="test-curl-7500-140135897994544"
Jul 29 20:21:27.177675 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.177653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:27.181429 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.181403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:27.181816 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.181799    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sqw7\" (UniqueName: \"kubernetes.io/projected/4f072d64-7b8b-4bcd-a769-8a7026a448e6-kube-api-access-5sqw7\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:21:27.283049 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.282983    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5hhd\" (UniqueName: \"kubernetes.io/projected/4a0ba544-ac51-4cfe-9335-ef365ecfd871-kube-api-access-q5hhd\") pod \"test-curl-7500-140135897970704\" (UID: \"4a0ba544-ac51-4cfe-9335-ef365ecfd871\") " pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:27.384036 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.383975    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5hhd\" (UniqueName: \"kubernetes.io/projected/4a0ba544-ac51-4cfe-9335-ef365ecfd871-kube-api-access-q5hhd\") pod \"test-curl-7500-140135897970704\" (UID: \"4a0ba544-ac51-4cfe-9335-ef365ecfd871\") " pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:27.392228 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.392203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5hhd\" (UniqueName: \"kubernetes.io/projected/4a0ba544-ac51-4cfe-9335-ef365ecfd871-kube-api-access-q5hhd\") pod \"test-curl-7500-140135897970704\" (UID: \"4a0ba544-ac51-4cfe-9335-ef365ecfd871\") " pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:27.488698 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.488653    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:27.633543 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.633513    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:27.635204 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:21:27.635131    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a0ba544_ac51_4cfe_9335_ef365ecfd871.slice/crio-23a99cb1c0f927351687b75738e199924c87eb68a7464a9b9b1c4b9b0e1738c3 WatchSource:0}: Error finding container 23a99cb1c0f927351687b75738e199924c87eb68a7464a9b9b1c4b9b0e1738c3: Status 404 returned error can't find the container with id 23a99cb1c0f927351687b75738e199924c87eb68a7464a9b9b1c4b9b0e1738c3
Jul 29 20:21:27.828284 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.828255    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:27.828487 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.828290    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0ae9aa369e2627393dda83e923f67a305482e61abbf0cf61e6bf7fe14fa24419"
Jul 29 20:21:27.829604 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:27.829578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897970704" event={"ID":"4a0ba544-ac51-4cfe-9335-ef365ecfd871","Type":"ContainerStarted","Data":"23a99cb1c0f927351687b75738e199924c87eb68a7464a9b9b1c4b9b0e1738c3"}
Jul 29 20:21:27.838684 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:21:27.838657    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7500-140135897994544\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:28.716100 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:21:28.716050    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7500-140135897994544\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" pod="opendatahub/test-curl-7500-140135897994544"
Jul 29 20:21:28.718779 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:28.718755    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f072d64-7b8b-4bcd-a769-8a7026a448e6" path="/var/lib/kubelet/pods/4f072d64-7b8b-4bcd-a769-8a7026a448e6/volumes"
Jul 29 20:21:28.836929 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:28.836891    2580 generic.go:356] "Generic (PLEG): container finished" podID="4a0ba544-ac51-4cfe-9335-ef365ecfd871" containerID="df1e9a255c72be26adc67ee4d2a8139288741223e2e9dca5fee561e7a8321e8d" exitCode=0
Jul 29 20:21:28.837137 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:28.836982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897970704" event={"ID":"4a0ba544-ac51-4cfe-9335-ef365ecfd871","Type":"ContainerDied","Data":"df1e9a255c72be26adc67ee4d2a8139288741223e2e9dca5fee561e7a8321e8d"}
Jul 29 20:21:29.801192 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:29.801142    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:21:29.983587 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:29.983555    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.004460 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.004432    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7500-140135897970704_4a0ba544-ac51-4cfe-9335-ef365ecfd871/test-curl-7500-140135897970704/0.log"
Jul 29 20:21:30.030949 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.030878    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:30.033381 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.033356    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:30.112405 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.112320    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5hhd\" (UniqueName: \"kubernetes.io/projected/4a0ba544-ac51-4cfe-9335-ef365ecfd871-kube-api-access-q5hhd\") pod \"4a0ba544-ac51-4cfe-9335-ef365ecfd871\" (UID: \"4a0ba544-ac51-4cfe-9335-ef365ecfd871\") "
Jul 29 20:21:30.114601 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.114574    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a0ba544-ac51-4cfe-9335-ef365ecfd871-kube-api-access-q5hhd" (OuterVolumeSpecName: "kube-api-access-q5hhd") pod "4a0ba544-ac51-4cfe-9335-ef365ecfd871" (UID: "4a0ba544-ac51-4cfe-9335-ef365ecfd871"). InnerVolumeSpecName "kube-api-access-q5hhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:21:30.152228 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.152198    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:30.152688 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.152673    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a0ba544-ac51-4cfe-9335-ef365ecfd871" containerName="test-curl-7500-140135897970704"
Jul 29 20:21:30.152737 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.152690    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a0ba544-ac51-4cfe-9335-ef365ecfd871" containerName="test-curl-7500-140135897970704"
Jul 29 20:21:30.152796 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.152787    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a0ba544-ac51-4cfe-9335-ef365ecfd871" containerName="test-curl-7500-140135897970704"
Jul 29 20:21:30.156074 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.156057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.159875 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.159846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:30.214095 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.214064    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5hhd\" (UniqueName: \"kubernetes.io/projected/4a0ba544-ac51-4cfe-9335-ef365ecfd871-kube-api-access-q5hhd\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:21:30.315327 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.315276    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcmlj\" (UniqueName: \"kubernetes.io/projected/805de7ea-8be0-4114-99e1-9d3aeede08fa-kube-api-access-fcmlj\") pod \"test-curl-7500-140135897970704\" (UID: \"805de7ea-8be0-4114-99e1-9d3aeede08fa\") " pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.416779 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.416744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcmlj\" (UniqueName: \"kubernetes.io/projected/805de7ea-8be0-4114-99e1-9d3aeede08fa-kube-api-access-fcmlj\") pod \"test-curl-7500-140135897970704\" (UID: \"805de7ea-8be0-4114-99e1-9d3aeede08fa\") " pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.426047 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.426019    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcmlj\" (UniqueName: \"kubernetes.io/projected/805de7ea-8be0-4114-99e1-9d3aeede08fa-kube-api-access-fcmlj\") pod \"test-curl-7500-140135897970704\" (UID: \"805de7ea-8be0-4114-99e1-9d3aeede08fa\") " pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.466089 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.466036    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.604030 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.603982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:30.606058 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:21:30.606026    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod805de7ea_8be0_4114_99e1_9d3aeede08fa.slice/crio-c629f66d2b7c6294d46c729babf8e3b49c82bc95ce7648ac6be6836b0dc13e26 WatchSource:0}: Error finding container c629f66d2b7c6294d46c729babf8e3b49c82bc95ce7648ac6be6836b0dc13e26: Status 404 returned error can't find the container with id c629f66d2b7c6294d46c729babf8e3b49c82bc95ce7648ac6be6836b0dc13e26
Jul 29 20:21:30.716874 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.716785    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a0ba544-ac51-4cfe-9335-ef365ecfd871" path="/var/lib/kubelet/pods/4a0ba544-ac51-4cfe-9335-ef365ecfd871/volumes"
Jul 29 20:21:30.849639 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.849599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897970704" event={"ID":"805de7ea-8be0-4114-99e1-9d3aeede08fa","Type":"ContainerStarted","Data":"c629f66d2b7c6294d46c729babf8e3b49c82bc95ce7648ac6be6836b0dc13e26"}
Jul 29 20:21:30.850836 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.850817    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:30.850968 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:30.850875    2580 scope.go:117] "RemoveContainer" containerID="df1e9a255c72be26adc67ee4d2a8139288741223e2e9dca5fee561e7a8321e8d"
Jul 29 20:21:31.857559 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:31.857523    2580 generic.go:356] "Generic (PLEG): container finished" podID="805de7ea-8be0-4114-99e1-9d3aeede08fa" containerID="5145cfaeeb60a575dfd9a16651f52f9a7aa69905e2214374a786e0ca4b3c8d0d" exitCode=0
Jul 29 20:21:31.857972 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:31.857605    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897970704" event={"ID":"805de7ea-8be0-4114-99e1-9d3aeede08fa","Type":"ContainerDied","Data":"5145cfaeeb60a575dfd9a16651f52f9a7aa69905e2214374a786e0ca4b3c8d0d"}
Jul 29 20:21:33.006084 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.006058    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:33.026756 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.026724    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7500-140135897970704_805de7ea-8be0-4114-99e1-9d3aeede08fa/test-curl-7500-140135897970704/0.log"
Jul 29 20:21:33.050488 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.050453    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:33.052605 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.052579    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7500-140135897970704"]
Jul 29 20:21:33.142375 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.142330    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcmlj\" (UniqueName: \"kubernetes.io/projected/805de7ea-8be0-4114-99e1-9d3aeede08fa-kube-api-access-fcmlj\") pod \"805de7ea-8be0-4114-99e1-9d3aeede08fa\" (UID: \"805de7ea-8be0-4114-99e1-9d3aeede08fa\") "
Jul 29 20:21:33.144596 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.144571    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/805de7ea-8be0-4114-99e1-9d3aeede08fa-kube-api-access-fcmlj" (OuterVolumeSpecName: "kube-api-access-fcmlj") pod "805de7ea-8be0-4114-99e1-9d3aeede08fa" (UID: "805de7ea-8be0-4114-99e1-9d3aeede08fa"). InnerVolumeSpecName "kube-api-access-fcmlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:21:33.178166 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.178132    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:33.178594 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.178581    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="805de7ea-8be0-4114-99e1-9d3aeede08fa" containerName="test-curl-7500-140135897970704"
Jul 29 20:21:33.178649 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.178595    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="805de7ea-8be0-4114-99e1-9d3aeede08fa" containerName="test-curl-7500-140135897970704"
Jul 29 20:21:33.178687 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.178663    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="805de7ea-8be0-4114-99e1-9d3aeede08fa" containerName="test-curl-7500-140135897970704"
Jul 29 20:21:33.181689 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.181673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:33.188313 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.188283    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:33.243557 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.243525    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcmlj\" (UniqueName: \"kubernetes.io/projected/805de7ea-8be0-4114-99e1-9d3aeede08fa-kube-api-access-fcmlj\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:21:33.345087 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.345049    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngmw8\" (UniqueName: \"kubernetes.io/projected/33ef9fc6-423d-4534-a22c-0b1cd677c193-kube-api-access-ngmw8\") pod \"test-curl-7500-140135897971712\" (UID: \"33ef9fc6-423d-4534-a22c-0b1cd677c193\") " pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:33.446630 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.446527    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngmw8\" (UniqueName: \"kubernetes.io/projected/33ef9fc6-423d-4534-a22c-0b1cd677c193-kube-api-access-ngmw8\") pod \"test-curl-7500-140135897971712\" (UID: \"33ef9fc6-423d-4534-a22c-0b1cd677c193\") " pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:33.454819 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.454789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngmw8\" (UniqueName: \"kubernetes.io/projected/33ef9fc6-423d-4534-a22c-0b1cd677c193-kube-api-access-ngmw8\") pod \"test-curl-7500-140135897971712\" (UID: \"33ef9fc6-423d-4534-a22c-0b1cd677c193\") " pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:33.492288 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.492252    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:33.629520 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.629488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:33.631213 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:21:33.631185    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod33ef9fc6_423d_4534_a22c_0b1cd677c193.slice/crio-2014cf1f2cec73ed07afcbadb19b506b1a45de3d6a40a917c2d27ee66484bfa3 WatchSource:0}: Error finding container 2014cf1f2cec73ed07afcbadb19b506b1a45de3d6a40a917c2d27ee66484bfa3: Status 404 returned error can't find the container with id 2014cf1f2cec73ed07afcbadb19b506b1a45de3d6a40a917c2d27ee66484bfa3
Jul 29 20:21:33.871318 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.871223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897971712" event={"ID":"33ef9fc6-423d-4534-a22c-0b1cd677c193","Type":"ContainerStarted","Data":"2014cf1f2cec73ed07afcbadb19b506b1a45de3d6a40a917c2d27ee66484bfa3"}
Jul 29 20:21:33.872474 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.872454    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:33.872605 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:33.872460    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c629f66d2b7c6294d46c729babf8e3b49c82bc95ce7648ac6be6836b0dc13e26"
Jul 29 20:21:33.882648 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:21:33.882620    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7500-140135897970704\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="805de7ea-8be0-4114-99e1-9d3aeede08fa" pod="opendatahub/test-curl-7500-140135897970704"
Jul 29 20:21:34.717706 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:34.717670    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="805de7ea-8be0-4114-99e1-9d3aeede08fa" path="/var/lib/kubelet/pods/805de7ea-8be0-4114-99e1-9d3aeede08fa/volumes"
Jul 29 20:21:34.879245 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:34.879211    2580 generic.go:356] "Generic (PLEG): container finished" podID="33ef9fc6-423d-4534-a22c-0b1cd677c193" containerID="a0fe4756da1bbed221025af8d239c0cf714ac80e88f636c8351ca6e5f1708825" exitCode=0
Jul 29 20:21:34.879441 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:34.879302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897971712" event={"ID":"33ef9fc6-423d-4534-a22c-0b1cd677c193","Type":"ContainerDied","Data":"a0fe4756da1bbed221025af8d239c0cf714ac80e88f636c8351ca6e5f1708825"}
Jul 29 20:21:36.030813 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.030784    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.049798 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.049766    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7500-140135897971712_33ef9fc6-423d-4534-a22c-0b1cd677c193/test-curl-7500-140135897971712/0.log"
Jul 29 20:21:36.069141 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.069111    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngmw8\" (UniqueName: \"kubernetes.io/projected/33ef9fc6-423d-4534-a22c-0b1cd677c193-kube-api-access-ngmw8\") pod \"33ef9fc6-423d-4534-a22c-0b1cd677c193\" (UID: \"33ef9fc6-423d-4534-a22c-0b1cd677c193\") "
Jul 29 20:21:36.071430 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.071394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33ef9fc6-423d-4534-a22c-0b1cd677c193-kube-api-access-ngmw8" (OuterVolumeSpecName: "kube-api-access-ngmw8") pod "33ef9fc6-423d-4534-a22c-0b1cd677c193" (UID: "33ef9fc6-423d-4534-a22c-0b1cd677c193"). InnerVolumeSpecName "kube-api-access-ngmw8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:21:36.078146 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.078117    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:36.080628 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.080605    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:36.169913 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.169813    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngmw8\" (UniqueName: \"kubernetes.io/projected/33ef9fc6-423d-4534-a22c-0b1cd677c193-kube-api-access-ngmw8\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:21:36.197786 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.197743    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:36.198265 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.198249    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33ef9fc6-423d-4534-a22c-0b1cd677c193" containerName="test-curl-7500-140135897971712"
Jul 29 20:21:36.198311 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.198268    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33ef9fc6-423d-4534-a22c-0b1cd677c193" containerName="test-curl-7500-140135897971712"
Jul 29 20:21:36.198352 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.198342    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33ef9fc6-423d-4534-a22c-0b1cd677c193" containerName="test-curl-7500-140135897971712"
Jul 29 20:21:36.201462 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.201444    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.205078 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.205054    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:36.270823 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.270790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pnxl\" (UniqueName: \"kubernetes.io/projected/8532f228-4fea-464e-817f-99e1b8312e19-kube-api-access-9pnxl\") pod \"test-curl-7500-140135897971712\" (UID: \"8532f228-4fea-464e-817f-99e1b8312e19\") " pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.371413 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.371366    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pnxl\" (UniqueName: \"kubernetes.io/projected/8532f228-4fea-464e-817f-99e1b8312e19-kube-api-access-9pnxl\") pod \"test-curl-7500-140135897971712\" (UID: \"8532f228-4fea-464e-817f-99e1b8312e19\") " pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.379950 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.379923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pnxl\" (UniqueName: \"kubernetes.io/projected/8532f228-4fea-464e-817f-99e1b8312e19-kube-api-access-9pnxl\") pod \"test-curl-7500-140135897971712\" (UID: \"8532f228-4fea-464e-817f-99e1b8312e19\") " pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.511453 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.511360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.652759 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.652651    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:36.721348 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.721309    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33ef9fc6-423d-4534-a22c-0b1cd677c193" path="/var/lib/kubelet/pods/33ef9fc6-423d-4534-a22c-0b1cd677c193/volumes"
Jul 29 20:21:36.892204 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.892170    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:36.892371 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.892214    2580 scope.go:117] "RemoveContainer" containerID="a0fe4756da1bbed221025af8d239c0cf714ac80e88f636c8351ca6e5f1708825"
Jul 29 20:21:36.893611 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:36.893474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897971712" event={"ID":"8532f228-4fea-464e-817f-99e1b8312e19","Type":"ContainerStarted","Data":"af9299d929853f5e72c1f6853b91ba88940613a162264f0faa51b0bd01c0a48c"}
Jul 29 20:21:37.901639 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:37.901610    2580 generic.go:356] "Generic (PLEG): container finished" podID="8532f228-4fea-464e-817f-99e1b8312e19" containerID="ad74e7eab03ee141b4b4668ca70c5e67cd0c74ab16044d033ae189959ed2b25a" exitCode=0
Jul 29 20:21:37.902114 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:37.901696    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7500-140135897971712" event={"ID":"8532f228-4fea-464e-817f-99e1b8312e19","Type":"ContainerDied","Data":"ad74e7eab03ee141b4b4668ca70c5e67cd0c74ab16044d033ae189959ed2b25a"}
Jul 29 20:21:39.072563 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.070613    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:39.090537 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.090506    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7500-140135897971712_8532f228-4fea-464e-817f-99e1b8312e19/test-curl-7500-140135897971712/0.log"
Jul 29 20:21:39.097586 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.097560    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9pnxl\" (UniqueName: \"kubernetes.io/projected/8532f228-4fea-464e-817f-99e1b8312e19-kube-api-access-9pnxl\") pod \"8532f228-4fea-464e-817f-99e1b8312e19\" (UID: \"8532f228-4fea-464e-817f-99e1b8312e19\") "
Jul 29 20:21:39.101286 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.101230    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8532f228-4fea-464e-817f-99e1b8312e19-kube-api-access-9pnxl" (OuterVolumeSpecName: "kube-api-access-9pnxl") pod "8532f228-4fea-464e-817f-99e1b8312e19" (UID: "8532f228-4fea-464e-817f-99e1b8312e19"). InnerVolumeSpecName "kube-api-access-9pnxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:21:39.111092 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.111066    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:39.114485 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.114461    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7500-140135897971712"]
Jul 29 20:21:39.199045 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.198942    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9pnxl\" (UniqueName: \"kubernetes.io/projected/8532f228-4fea-464e-817f-99e1b8312e19-kube-api-access-9pnxl\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:21:39.610117 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.610034    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:39.916637 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.916612    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af9299d929853f5e72c1f6853b91ba88940613a162264f0faa51b0bd01c0a48c"
Jul 29 20:21:39.916831 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.916647    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:39.919973 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:39.919947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:39.929040 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:21:39.928983    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7500-140135897971712\" is forbidden: User \"system:node:ip-10-0-140-87.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-87.ec2.internal' and this object" podUID="8532f228-4fea-464e-817f-99e1b8312e19" pod="opendatahub/test-curl-7500-140135897971712"
Jul 29 20:21:40.080057 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:40.080024    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:40.717665 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:40.717620    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8532f228-4fea-464e-817f-99e1b8312e19" path="/var/lib/kubelet/pods/8532f228-4fea-464e-817f-99e1b8312e19/volumes"
Jul 29 20:21:42.530429 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.530387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:42.625646 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.625606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:42.703440 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.703408    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"]
Jul 29 20:21:42.703864 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.703852    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8532f228-4fea-464e-817f-99e1b8312e19" containerName="test-curl-7500-140135897971712"
Jul 29 20:21:42.703919 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.703866    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8532f228-4fea-464e-817f-99e1b8312e19" containerName="test-curl-7500-140135897971712"
Jul 29 20:21:42.703977 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.703965    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8532f228-4fea-464e-817f-99e1b8312e19" containerName="test-curl-7500-140135897971712"
Jul 29 20:21:42.709235 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.709210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.711026 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.710978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-658cad-serving-cert\""
Jul 29 20:21:42.711197 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.711171    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-wzkl2\""
Jul 29 20:21:42.718827 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.718799    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"]
Jul 29 20:21:42.732085 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.732050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.732233 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.732164    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82l82\" (UniqueName: \"kubernetes.io/projected/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-kube-api-access-82l82\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.732280 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.732227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.735331 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.735298    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:42.833163 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.833067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.833163 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.833123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82l82\" (UniqueName: \"kubernetes.io/projected/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-kube-api-access-82l82\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.833163 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.833146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.833430 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:21:42.833243    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-serving-cert: secret "maas-api-e2e-ipp-a-658cad-serving-cert" not found
Jul 29 20:21:42.833430 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:21:42.833301    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls podName:62924f7c-d833-4bea-ac0d-ec24cafa2e8a nodeName:}" failed. No retries permitted until 2026-07-29 20:21:43.333284632 +0000 UTC m=+3125.211138938 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls") pod "maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" (UID: "62924f7c-d833-4bea-ac0d-ec24cafa2e8a") : secret "maas-api-e2e-ipp-a-658cad-serving-cert" not found
Jul 29 20:21:42.833865 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.833844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:42.843952 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:42.843921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82l82\" (UniqueName: \"kubernetes.io/projected/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-kube-api-access-82l82\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:43.069098 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.068972    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:43.338204 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.338163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:43.340946 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.340919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls\") pod \"maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:43.345983 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.345958    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:43.448085 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.448042    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:21:43.624209 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.624124    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:43.775179 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.775149    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"]
Jul 29 20:21:43.776954 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:21:43.776919    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62924f7c_d833_4bea_ac0d_ec24cafa2e8a.slice/crio-59ae8fa1e1944a4c6897a6d566bc79b74e0de66d25a47775dcef8098f1949fbb WatchSource:0}: Error finding container 59ae8fa1e1944a4c6897a6d566bc79b74e0de66d25a47775dcef8098f1949fbb: Status 404 returned error can't find the container with id 59ae8fa1e1944a4c6897a6d566bc79b74e0de66d25a47775dcef8098f1949fbb
Jul 29 20:21:43.941869 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:43.941831    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" event={"ID":"62924f7c-d833-4bea-ac0d-ec24cafa2e8a","Type":"ContainerStarted","Data":"59ae8fa1e1944a4c6897a6d566bc79b74e0de66d25a47775dcef8098f1949fbb"}
Jul 29 20:21:46.964564 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:46.964525    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" event={"ID":"62924f7c-d833-4bea-ac0d-ec24cafa2e8a","Type":"ContainerStarted","Data":"46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4"}
Jul 29 20:21:46.964967 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:46.964593    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:21:46.982932 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:46.982875    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" podStartSLOduration=2.518341542 podStartE2EDuration="4.982858076s" totalImagesPullingTime="2.464516534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:21:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:21:46.980819314 +0000 UTC m=+3128.858673642" watchObservedRunningTime="2026-07-29 20:21:46.982858076 +0000 UTC m=+3128.860712439"
Jul 29 20:21:50.897781 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:50.897744    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:21:52.976455 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:21:52.976423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:22:06.571194 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.571158    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qdtq2"]
Jul 29 20:22:06.576687 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.576660    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:06.582012 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.581972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qdtq2"]
Jul 29 20:22:06.688905 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.688866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftk9x\" (UniqueName: \"kubernetes.io/projected/6659b603-3fde-41c0-b61d-518b04736db5-kube-api-access-ftk9x\") pod \"kuadrant-operator-catalog-qdtq2\" (UID: \"6659b603-3fde-41c0-b61d-518b04736db5\") " pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:06.789663 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.789627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftk9x\" (UniqueName: \"kubernetes.io/projected/6659b603-3fde-41c0-b61d-518b04736db5-kube-api-access-ftk9x\") pod \"kuadrant-operator-catalog-qdtq2\" (UID: \"6659b603-3fde-41c0-b61d-518b04736db5\") " pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:06.798299 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.798277    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftk9x\" (UniqueName: \"kubernetes.io/projected/6659b603-3fde-41c0-b61d-518b04736db5-kube-api-access-ftk9x\") pod \"kuadrant-operator-catalog-qdtq2\" (UID: \"6659b603-3fde-41c0-b61d-518b04736db5\") " pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:06.890170 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:06.890135    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:07.038490 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:07.038461    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qdtq2"]
Jul 29 20:22:07.040731 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:22:07.040703    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6659b603_3fde_41c0_b61d_518b04736db5.slice/crio-1594f038a912b0057763a7d038a343086583e53ee38136c8181af6afde1baff9 WatchSource:0}: Error finding container 1594f038a912b0057763a7d038a343086583e53ee38136c8181af6afde1baff9: Status 404 returned error can't find the container with id 1594f038a912b0057763a7d038a343086583e53ee38136c8181af6afde1baff9
Jul 29 20:22:07.081649 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:07.081603    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2" event={"ID":"6659b603-3fde-41c0-b61d-518b04736db5","Type":"ContainerStarted","Data":"1594f038a912b0057763a7d038a343086583e53ee38136c8181af6afde1baff9"}
Jul 29 20:22:07.489602 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:07.489565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:08.089520 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:08.089477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2" event={"ID":"6659b603-3fde-41c0-b61d-518b04736db5","Type":"ContainerStarted","Data":"d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8"}
Jul 29 20:22:08.105730 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:08.105678    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2" podStartSLOduration=1.5645927670000002 podStartE2EDuration="2.105662485s" totalImagesPullingTime="541.069718ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:22:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:22:08.104371979 +0000 UTC m=+3149.982226305" watchObservedRunningTime="2026-07-29 20:22:08.105662485 +0000 UTC m=+3149.983516813"
Jul 29 20:22:08.135007 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:08.134972    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:08.444256 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:08.444194    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:08.623897 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:08.623860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:10.477311 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:10.477275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:10.534113 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:10.534081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:11.699324 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:11.699291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:11.783623 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:11.783578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:11.849204 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:11.849168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:12.412081 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:12.412046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:12.514324 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:12.514290    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:13.254196 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:13.254164    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:22:16.891361 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:16.891311    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:16.891782 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:16.891383    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:16.914117 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:16.914088    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:17.163301 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:17.163214    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:17.200595 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:17.200560    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qdtq2"]
Jul 29 20:22:19.152662 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:19.152604    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2" podUID="6659b603-3fde-41c0-b61d-518b04736db5" containerName="registry-server" containerID="cri-o://d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8" gracePeriod=2
Jul 29 20:22:19.409510 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:19.409445    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:19.510815 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:19.510780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftk9x\" (UniqueName: \"kubernetes.io/projected/6659b603-3fde-41c0-b61d-518b04736db5-kube-api-access-ftk9x\") pod \"6659b603-3fde-41c0-b61d-518b04736db5\" (UID: \"6659b603-3fde-41c0-b61d-518b04736db5\") "
Jul 29 20:22:19.513125 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:19.513094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6659b603-3fde-41c0-b61d-518b04736db5-kube-api-access-ftk9x" (OuterVolumeSpecName: "kube-api-access-ftk9x") pod "6659b603-3fde-41c0-b61d-518b04736db5" (UID: "6659b603-3fde-41c0-b61d-518b04736db5"). InnerVolumeSpecName "kube-api-access-ftk9x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:22:19.612296 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:19.612261    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftk9x\" (UniqueName: \"kubernetes.io/projected/6659b603-3fde-41c0-b61d-518b04736db5-kube-api-access-ftk9x\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:22:20.159764 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.159730    2580 generic.go:356] "Generic (PLEG): container finished" podID="6659b603-3fde-41c0-b61d-518b04736db5" containerID="d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8" exitCode=0
Jul 29 20:22:20.160200 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.159774    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2"
Jul 29 20:22:20.160200 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.159781    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2" event={"ID":"6659b603-3fde-41c0-b61d-518b04736db5","Type":"ContainerDied","Data":"d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8"}
Jul 29 20:22:20.160200 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.159825    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-qdtq2" event={"ID":"6659b603-3fde-41c0-b61d-518b04736db5","Type":"ContainerDied","Data":"1594f038a912b0057763a7d038a343086583e53ee38136c8181af6afde1baff9"}
Jul 29 20:22:20.160200 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.159846    2580 scope.go:117] "RemoveContainer" containerID="d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8"
Jul 29 20:22:20.170744 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.170723    2580 scope.go:117] "RemoveContainer" containerID="d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8"
Jul 29 20:22:20.171041 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:22:20.170986    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8\": container with ID starting with d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8 not found: ID does not exist" containerID="d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8"
Jul 29 20:22:20.171141 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.171050    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8"} err="failed to get container status \"d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8\": rpc error: code = NotFound desc = could not find container \"d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8\": container with ID starting with d60b7f8552fe707e8b377a28a46e521d003d27306a61930fd8f917d059bffad8 not found: ID does not exist"
Jul 29 20:22:20.182218 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.182192    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qdtq2"]
Jul 29 20:22:20.184279 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.184259    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-qdtq2"]
Jul 29 20:22:20.716971 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:20.716931    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6659b603-3fde-41c0-b61d-518b04736db5" path="/var/lib/kubelet/pods/6659b603-3fde-41c0-b61d-518b04736db5/volumes"
Jul 29 20:22:21.601797 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:21.601761    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:22:36.692712 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:22:36.692669    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:04.990207 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:04.990173    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:08.800159 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:08.800122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:23:34.896714 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:34.896679    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:23:46.425819 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.425776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:46.699317 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.699227    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"]
Jul 29 20:23:46.699762 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.699738    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6659b603-3fde-41c0-b61d-518b04736db5" containerName="registry-server"
Jul 29 20:23:46.699762 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.699758    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6659b603-3fde-41c0-b61d-518b04736db5" containerName="registry-server"
Jul 29 20:23:46.699934 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.699846    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6659b603-3fde-41c0-b61d-518b04736db5" containerName="registry-server"
Jul 29 20:23:46.703562 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.703543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.705294 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.705273    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-f4fc66-openshift-default-dockercfg-mdscl\""
Jul 29 20:23:46.719039 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.719011    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"]
Jul 29 20:23:46.727227 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.727200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:46.756982 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.756948    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9wjj\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-kube-api-access-d9wjj\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.756982 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.756984    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29997415-ff38-4059-bfd9-597b8d418a60-istio-podinfo\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757220 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757125    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-credential-socket\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757286 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757227    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29997415-ff38-4059-bfd9-597b8d418a60-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757341 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-istio-token\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757409 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-socket\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757464 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-data\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757507 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757470    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-certs\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.757554 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.757521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-envoy\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858188 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29997415-ff38-4059-bfd9-597b8d418a60-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-istio-token\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-socket\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-data\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-certs\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858304    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-envoy\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858339    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9wjj\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-kube-api-access-d9wjj\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858370    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29997415-ff38-4059-bfd9-597b8d418a60-istio-podinfo\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858778 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-credential-socket\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858840 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858795    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-credential-socket\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858840 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-socket\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.858948 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.858885    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-data\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.859049 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.859016    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-certs\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.859117 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.859041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29997415-ff38-4059-bfd9-597b8d418a60-istiod-ca-cert\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.860821 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.860798    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-envoy\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.861133 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.861111    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29997415-ff38-4059-bfd9-597b8d418a60-istio-podinfo\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.868587 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.868562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-istio-token\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.868918 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.868898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9wjj\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-kube-api-access-d9wjj\") pod \"e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") " pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:46.911850 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:46.911817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:47.014628 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.014518    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:47.164650 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.164617    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"]
Jul 29 20:23:47.166607 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:23:47.166582    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29997415_ff38_4059_bfd9_597b8d418a60.slice/crio-b9fcc59b2b10a33e1b2f5d210836c847e5eba005d47e7e663a809dd358990376 WatchSource:0}: Error finding container b9fcc59b2b10a33e1b2f5d210836c847e5eba005d47e7e663a809dd358990376: Status 404 returned error can't find the container with id b9fcc59b2b10a33e1b2f5d210836c847e5eba005d47e7e663a809dd358990376
Jul 29 20:23:47.169286 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.169255    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 20:23:47.169369 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.169323    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 20:23:47.169369 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.169350    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 20:23:47.659146 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.659107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" event={"ID":"29997415-ff38-4059-bfd9-597b8d418a60","Type":"ContainerStarted","Data":"726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"}
Jul 29 20:23:47.659533 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.659177    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" event={"ID":"29997415-ff38-4059-bfd9-597b8d418a60","Type":"ContainerStarted","Data":"b9fcc59b2b10a33e1b2f5d210836c847e5eba005d47e7e663a809dd358990376"}
Jul 29 20:23:47.679680 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:47.679628    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" podStartSLOduration=1.679614338 podStartE2EDuration="1.679614338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:23:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:23:47.678036526 +0000 UTC m=+3249.555890854" watchObservedRunningTime="2026-07-29 20:23:47.679614338 +0000 UTC m=+3249.557468665"
Jul 29 20:23:48.015180 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:48.015084    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:48.020935 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:48.020907    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:48.666477 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:48.666437    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:48.667492 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:48.667470    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:23:49.603583 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.603551    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:49.691435 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.691398    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"]
Jul 29 20:23:49.695116 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.695091    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.696857 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.696838    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-f4fc66-serving-cert\""
Jul 29 20:23:49.707270 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.707247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"]
Jul 29 20:23:49.786961 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.786912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4svsd\" (UniqueName: \"kubernetes.io/projected/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-kube-api-access-4svsd\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.787163 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.787028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.787163 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.787112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.888453 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.888420    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.888609 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.888485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.888609 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.888536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4svsd\" (UniqueName: \"kubernetes.io/projected/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-kube-api-access-4svsd\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.889231 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.889201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.891524 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.891499    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:49.900601 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:49.900574    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4svsd\" (UniqueName: \"kubernetes.io/projected/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-kube-api-access-4svsd\") pod \"maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:50.008509 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:50.008466    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:50.162016 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:50.161948    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"]
Jul 29 20:23:50.679279 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:50.679241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws" event={"ID":"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c","Type":"ContainerStarted","Data":"a301533bccc8989966af918caba1cd6dbc565e5c3289a7294f461f31d8e8a002"}
Jul 29 20:23:50.847136 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:50.847099    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:23:51.687820 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:51.687786    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws" event={"ID":"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c","Type":"ContainerStarted","Data":"7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698"}
Jul 29 20:23:51.688159 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:51.687914    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:23:51.705652 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:51.705596    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws" podStartSLOduration=2.172796499 podStartE2EDuration="2.705582277s" totalImagesPullingTime="532.785778ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:23:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:23:51.702625543 +0000 UTC m=+3253.580479873" watchObservedRunningTime="2026-07-29 20:23:51.705582277 +0000 UTC m=+3253.583436605"
Jul 29 20:23:54.092650 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:54.092611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:23:57.700898 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:23:57.700869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:24:02.777379 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:02.777286    2580 generic.go:356] "Generic (PLEG): container finished" podID="29997415-ff38-4059-bfd9-597b8d418a60" containerID="726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d" exitCode=0
Jul 29 20:24:02.777379 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:02.777359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" event={"ID":"29997415-ff38-4059-bfd9-597b8d418a60","Type":"ContainerDied","Data":"726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"}
Jul 29 20:24:02.777818 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:02.777768    2580 scope.go:117] "RemoveContainer" containerID="726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"
Jul 29 20:24:02.778788 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:02.778760    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 20:24:02.778837 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:02.778829    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 20:24:02.778870 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:02.778858    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"}
Jul 29 20:24:03.666784 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:03.666747    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:03.785944 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:03.785908    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" event={"ID":"29997415-ff38-4059-bfd9-597b8d418a60","Type":"ContainerStarted","Data":"b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e"}
Jul 29 20:24:04.015112 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:04.014983    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:04.016616 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:04.016580    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 29 20:24:04.016755 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:04.016651    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 29 20:24:05.016043 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:05.015976    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 29 20:24:05.016576 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:05.016072    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 29 20:24:06.016381 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:06.015858    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 29 20:24:06.016381 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:06.015942    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 29 20:24:07.015665 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:07.015619    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body=
Jul 29 20:24:07.015891 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:07.015697    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused"
Jul 29 20:24:08.019456 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:08.019426    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:08.019852 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:08.019692    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:08.020504 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:08.020486    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:14.553615 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:14.553564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:15.236007 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.235955    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"]
Jul 29 20:24:15.239882 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.239865    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:15.241694 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.241672    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:24:15.247051 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.247026    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"]
Jul 29 20:24:15.351058 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.350982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2f5d1c-eb7a-4679-91e2-edddc2edecab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8\" (UID: \"5b2f5d1c-eb7a-4679-91e2-edddc2edecab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:15.452024 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.451951    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2f5d1c-eb7a-4679-91e2-edddc2edecab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8\" (UID: \"5b2f5d1c-eb7a-4679-91e2-edddc2edecab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:15.452624 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.452602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2f5d1c-eb7a-4679-91e2-edddc2edecab-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8\" (UID: \"5b2f5d1c-eb7a-4679-91e2-edddc2edecab\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:15.551893 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.551801    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:15.695484 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.695445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"]
Jul 29 20:24:15.696837 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:24:15.696810    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b2f5d1c_eb7a_4679_91e2_edddc2edecab.slice/crio-81275c8b3fec2550c1d75f03a195ec1c2e69ffb572e5f1eec34635dcb236d73e WatchSource:0}: Error finding container 81275c8b3fec2550c1d75f03a195ec1c2e69ffb572e5f1eec34635dcb236d73e: Status 404 returned error can't find the container with id 81275c8b3fec2550c1d75f03a195ec1c2e69ffb572e5f1eec34635dcb236d73e
Jul 29 20:24:15.855666 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:15.855565    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8" event={"ID":"5b2f5d1c-eb7a-4679-91e2-edddc2edecab","Type":"ContainerStarted","Data":"81275c8b3fec2550c1d75f03a195ec1c2e69ffb572e5f1eec34635dcb236d73e"}
Jul 29 20:24:16.863417 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:16.863327    2580 generic.go:356] "Generic (PLEG): container finished" podID="5b2f5d1c-eb7a-4679-91e2-edddc2edecab" containerID="25a516a9a51060025f0a17a937b4ead94c1c9a3b8db036cb54670ae56e492c49" exitCode=0
Jul 29 20:24:16.863806 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:16.863416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8" event={"ID":"5b2f5d1c-eb7a-4679-91e2-edddc2edecab","Type":"ContainerDied","Data":"25a516a9a51060025f0a17a937b4ead94c1c9a3b8db036cb54670ae56e492c49"}
Jul 29 20:24:18.013625 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.013599    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:18.183107 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.183069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2f5d1c-eb7a-4679-91e2-edddc2edecab-maas-api-service-ca\") pod \"5b2f5d1c-eb7a-4679-91e2-edddc2edecab\" (UID: \"5b2f5d1c-eb7a-4679-91e2-edddc2edecab\") "
Jul 29 20:24:18.183522 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.183495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b2f5d1c-eb7a-4679-91e2-edddc2edecab-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5b2f5d1c-eb7a-4679-91e2-edddc2edecab" (UID: "5b2f5d1c-eb7a-4679-91e2-edddc2edecab"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:24:18.284205 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.284171    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5b2f5d1c-eb7a-4679-91e2-edddc2edecab-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:18.876356 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.876310    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"
Jul 29 20:24:18.876518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.876359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8" event={"ID":"5b2f5d1c-eb7a-4679-91e2-edddc2edecab","Type":"ContainerDied","Data":"81275c8b3fec2550c1d75f03a195ec1c2e69ffb572e5f1eec34635dcb236d73e"}
Jul 29 20:24:18.876518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:18.876415    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="81275c8b3fec2550c1d75f03a195ec1c2e69ffb572e5f1eec34635dcb236d73e"
Jul 29 20:24:20.402947 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.402911    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"]
Jul 29 20:24:20.403484 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.403192    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws" podUID="8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" containerName="maas-api" containerID="cri-o://7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698" gracePeriod=30
Jul 29 20:24:20.435071 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.435038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:20.504810 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.504779    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:20.566052 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.566011    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:20.618259 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.618230    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"]
Jul 29 20:24:20.621603 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.621576    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-f4fc66-js6w8"]
Jul 29 20:24:20.622705 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.622681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:20.672636 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.672611    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:24:20.706794 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.706766    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4svsd\" (UniqueName: \"kubernetes.io/projected/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-kube-api-access-4svsd\") pod \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") "
Jul 29 20:24:20.706989 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.706820    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-openshift-service-ca\") pod \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") "
Jul 29 20:24:20.706989 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.706943    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-maas-api-tls\") pod \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\" (UID: \"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c\") "
Jul 29 20:24:20.707249 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.707226    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" (UID: "8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:24:20.709233 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.709208    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" (UID: "8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:24:20.709320 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.709259    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-kube-api-access-4svsd" (OuterVolumeSpecName: "kube-api-access-4svsd") pod "8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" (UID: "8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c"). InnerVolumeSpecName "kube-api-access-4svsd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:24:20.721095 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.718259    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b2f5d1c-eb7a-4679-91e2-edddc2edecab" path="/var/lib/kubelet/pods/5b2f5d1c-eb7a-4679-91e2-edddc2edecab/volumes"
Jul 29 20:24:20.808580 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.808524    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-maas-api-tls\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:20.808580 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.808563    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4svsd\" (UniqueName: \"kubernetes.io/projected/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-kube-api-access-4svsd\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:20.808580 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.808579    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c-openshift-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:20.889371 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.889329    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" containerID="7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698" exitCode=0
Jul 29 20:24:20.889577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.889395    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"
Jul 29 20:24:20.889577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.889417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws" event={"ID":"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c","Type":"ContainerDied","Data":"7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698"}
Jul 29 20:24:20.889577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.889487    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws" event={"ID":"8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c","Type":"ContainerDied","Data":"a301533bccc8989966af918caba1cd6dbc565e5c3289a7294f461f31d8e8a002"}
Jul 29 20:24:20.889577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.889517    2580 scope.go:117] "RemoveContainer" containerID="7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698"
Jul 29 20:24:20.899167 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.899091    2580 scope.go:117] "RemoveContainer" containerID="7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698"
Jul 29 20:24:20.899342 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:24:20.899325    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698\": container with ID starting with 7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698 not found: ID does not exist" containerID="7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698"
Jul 29 20:24:20.899391 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.899350    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698"} err="failed to get container status \"7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698\": rpc error: code = NotFound desc = could not find container \"7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698\": container with ID starting with 7dce363544c238e38882c5a17292ab4c157b9551d28926b6930cda18e3c03698 not found: ID does not exist"
Jul 29 20:24:20.907432 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.907376    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"]
Jul 29 20:24:20.911737 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.911710    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-f4fc66-6fbc4bc665-hmtws"]
Jul 29 20:24:20.981093 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:20.981057    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:22.717141 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:22.717107    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" path="/var/lib/kubelet/pods/8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c/volumes"
Jul 29 20:24:24.399064 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:24.399027    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:24:26.929579 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:26.929531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:26.988937 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:26.988864    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:27.214517 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:27.214412    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"]
Jul 29 20:24:27.214763 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:27.214728    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy" containerID="cri-o://b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e" gracePeriod=30
Jul 29 20:24:27.238283 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:27.238240    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:27.459923 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:27.459875    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:30.811104 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811065    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"]
Jul 29 20:24:30.811533 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811520    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" containerName="maas-api"
Jul 29 20:24:30.811577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811534    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" containerName="maas-api"
Jul 29 20:24:30.811577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811544    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b2f5d1c-eb7a-4679-91e2-edddc2edecab" containerName="revoke-keys"
Jul 29 20:24:30.811577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811549    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b2f5d1c-eb7a-4679-91e2-edddc2edecab" containerName="revoke-keys"
Jul 29 20:24:30.811728 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811626    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a41e02a-d5cb-4cc0-98a7-cc0a4edebb1c" containerName="maas-api"
Jul 29 20:24:30.811728 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.811637    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b2f5d1c-eb7a-4679-91e2-edddc2edecab" containerName="revoke-keys"
Jul 29 20:24:30.814821 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.814803    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:30.816638 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.816618    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:24:30.821657 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.821635    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"]
Jul 29 20:24:30.907291 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:30.907253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d916b-09e7-4518-9503-0a83986eecaa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5\" (UID: \"ed8d916b-09e7-4518-9503-0a83986eecaa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:31.007922 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.007886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d916b-09e7-4518-9503-0a83986eecaa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5\" (UID: \"ed8d916b-09e7-4518-9503-0a83986eecaa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:31.008516 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.008495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d916b-09e7-4518-9503-0a83986eecaa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5\" (UID: \"ed8d916b-09e7-4518-9503-0a83986eecaa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:31.127679 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.127652    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:31.265970 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.265935    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"]
Jul 29 20:24:31.267560 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:24:31.267532    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded8d916b_09e7_4518_9503_0a83986eecaa.slice/crio-d7714d807e907673d0912759fac5c1874e4cefaff4ed42030c1bedf93664be93 WatchSource:0}: Error finding container d7714d807e907673d0912759fac5c1874e4cefaff4ed42030c1bedf93664be93: Status 404 returned error can't find the container with id d7714d807e907673d0912759fac5c1874e4cefaff4ed42030c1bedf93664be93
Jul 29 20:24:31.960023 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.959957    2580 generic.go:356] "Generic (PLEG): container finished" podID="ed8d916b-09e7-4518-9503-0a83986eecaa" containerID="2f7604d557d7029db611032715170ee0a63c46ebb0841c2fbb230dd71f105195" exitCode=0
Jul 29 20:24:31.960463 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.960043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5" event={"ID":"ed8d916b-09e7-4518-9503-0a83986eecaa","Type":"ContainerDied","Data":"2f7604d557d7029db611032715170ee0a63c46ebb0841c2fbb230dd71f105195"}
Jul 29 20:24:31.960463 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:31.960124    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5" event={"ID":"ed8d916b-09e7-4518-9503-0a83986eecaa","Type":"ContainerStarted","Data":"d7714d807e907673d0912759fac5c1874e4cefaff4ed42030c1bedf93664be93"}
Jul 29 20:24:32.481680 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.481615    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:32.522598 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522566    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-socket\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.522598 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522605    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9wjj\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-kube-api-access-d9wjj\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.522871 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-data\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.522871 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522831    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-certs\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.523009 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:24:32.523009 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522917    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29997415-ff38-4059-bfd9-597b8d418a60-istiod-ca-cert\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.523009 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522955    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-envoy\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.523225 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.522985    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29997415-ff38-4059-bfd9-597b8d418a60-istio-podinfo\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.523225 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-credential-socket\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.523225 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523127    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-istio-token\") pod \"29997415-ff38-4059-bfd9-597b8d418a60\" (UID: \"29997415-ff38-4059-bfd9-597b8d418a60\") "
Jul 29 20:24:32.523382 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29997415-ff38-4059-bfd9-597b8d418a60-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:24:32.523382 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523349    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:24:32.523515 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523488    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-data" (OuterVolumeSpecName: "istio-data") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:24:32.523726 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523704    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:24:32.523842 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523780    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-socket\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.523842 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523803    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-data\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.523842 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523818    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-workload-certs\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.523842 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.523831    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29997415-ff38-4059-bfd9-597b8d418a60-istiod-ca-cert\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.525701 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.525670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-kube-api-access-d9wjj" (OuterVolumeSpecName: "kube-api-access-d9wjj") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "kube-api-access-d9wjj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:24:32.525953 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.525922    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29997415-ff38-4059-bfd9-597b8d418a60-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 29 20:24:32.526100 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.526080    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-istio-token" (OuterVolumeSpecName: "istio-token") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:24:32.526178 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.526102    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29997415-ff38-4059-bfd9-597b8d418a60" (UID: "29997415-ff38-4059-bfd9-597b8d418a60"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 29 20:24:32.625122 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.625079    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9wjj\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-kube-api-access-d9wjj\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.625122 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.625114    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-istio-envoy\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.625122 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.625124    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29997415-ff38-4059-bfd9-597b8d418a60-istio-podinfo\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.625122 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.625132    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29997415-ff38-4059-bfd9-597b8d418a60-credential-socket\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.625393 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.625142    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29997415-ff38-4059-bfd9-597b8d418a60-istio-token\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:32.967391 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.967360    2580 generic.go:356] "Generic (PLEG): container finished" podID="29997415-ff38-4059-bfd9-597b8d418a60" containerID="b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e" exitCode=0
Jul 29 20:24:32.967826 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.967413    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"
Jul 29 20:24:32.967826 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.967440    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" event={"ID":"29997415-ff38-4059-bfd9-597b8d418a60","Type":"ContainerDied","Data":"b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e"}
Jul 29 20:24:32.967826 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.967489    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc" event={"ID":"29997415-ff38-4059-bfd9-597b8d418a60","Type":"ContainerDied","Data":"b9fcc59b2b10a33e1b2f5d210836c847e5eba005d47e7e663a809dd358990376"}
Jul 29 20:24:32.967826 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.967510    2580 scope.go:117] "RemoveContainer" containerID="b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e"
Jul 29 20:24:32.977733 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.977709    2580 scope.go:117] "RemoveContainer" containerID="726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"
Jul 29 20:24:32.984889 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.984861    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"]
Jul 29 20:24:32.988730 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.988705    2580 scope.go:117] "RemoveContainer" containerID="b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e"
Jul 29 20:24:32.989120 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:24:32.989089    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e\": container with ID starting with b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e not found: ID does not exist" containerID="b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e"
Jul 29 20:24:32.989210 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.989132    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e"} err="failed to get container status \"b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e\": rpc error: code = NotFound desc = could not find container \"b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e\": container with ID starting with b1f4a8743c6f4b49ddc44092de12811889f8251e9fa78ab9ff5f24f129810c1e not found: ID does not exist"
Jul 29 20:24:32.989210 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.989160    2580 scope.go:117] "RemoveContainer" containerID="726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"
Jul 29 20:24:32.989478 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:24:32.989456    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d\": container with ID starting with 726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d not found: ID does not exist" containerID="726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"
Jul 29 20:24:32.989542 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.989488    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d"} err="failed to get container status \"726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d\": rpc error: code = NotFound desc = could not find container \"726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d\": container with ID starting with 726d3c66ba952337cfbde499994af26ddf23ebeb244e050481258833fd7f550d not found: ID does not exist"
Jul 29 20:24:32.989586 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:32.989542    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-f4fc66-openshift-default-8444bc57bb-k96cc"]
Jul 29 20:24:33.099742 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.099715    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:33.231178 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.231092    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d916b-09e7-4518-9503-0a83986eecaa-maas-api-service-ca\") pod \"ed8d916b-09e7-4518-9503-0a83986eecaa\" (UID: \"ed8d916b-09e7-4518-9503-0a83986eecaa\") "
Jul 29 20:24:33.231456 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.231429    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ed8d916b-09e7-4518-9503-0a83986eecaa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ed8d916b-09e7-4518-9503-0a83986eecaa" (UID: "ed8d916b-09e7-4518-9503-0a83986eecaa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:24:33.332595 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.332554    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ed8d916b-09e7-4518-9503-0a83986eecaa-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:33.975097 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.975059    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"
Jul 29 20:24:33.975568 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.975094    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5" event={"ID":"ed8d916b-09e7-4518-9503-0a83986eecaa","Type":"ContainerDied","Data":"d7714d807e907673d0912759fac5c1874e4cefaff4ed42030c1bedf93664be93"}
Jul 29 20:24:33.975568 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:33.975140    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d7714d807e907673d0912759fac5c1874e4cefaff4ed42030c1bedf93664be93"
Jul 29 20:24:34.717021 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:34.716961    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29997415-ff38-4059-bfd9-597b8d418a60" path="/var/lib/kubelet/pods/29997415-ff38-4059-bfd9-597b8d418a60/volumes"
Jul 29 20:24:38.938251 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:38.938220    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:24:38.945179 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:38.945151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:24:38.947299 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:38.947282    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:24:38.953632 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:38.953611    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t6vcr_027102c3-1d0d-4c79-bdeb-0099468f8beb/ovn-acl-logging/0.log"
Jul 29 20:24:46.367295 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.367255    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"]
Jul 29 20:24:46.367790 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.367532    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" podUID="62924f7c-d833-4bea-ac0d-ec24cafa2e8a" containerName="maas-api" containerID="cri-o://46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4" gracePeriod=30
Jul 29 20:24:46.594153 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.594108    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"]
Jul 29 20:24:46.597536 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.597506    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-658cad-btkc5"]
Jul 29 20:24:46.654308 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.654281    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:24:46.719091 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.719055    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed8d916b-09e7-4518-9503-0a83986eecaa" path="/var/lib/kubelet/pods/ed8d916b-09e7-4518-9503-0a83986eecaa/volumes"
Jul 29 20:24:46.767827 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.767796    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82l82\" (UniqueName: \"kubernetes.io/projected/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-kube-api-access-82l82\") pod \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") "
Jul 29 20:24:46.768041 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.767850    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-openshift-service-ca\") pod \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") "
Jul 29 20:24:46.768041 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.767891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls\") pod \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\" (UID: \"62924f7c-d833-4bea-ac0d-ec24cafa2e8a\") "
Jul 29 20:24:46.768215 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.768192    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62924f7c-d833-4bea-ac0d-ec24cafa2e8a" (UID: "62924f7c-d833-4bea-ac0d-ec24cafa2e8a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:24:46.770067 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.770043    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-kube-api-access-82l82" (OuterVolumeSpecName: "kube-api-access-82l82") pod "62924f7c-d833-4bea-ac0d-ec24cafa2e8a" (UID: "62924f7c-d833-4bea-ac0d-ec24cafa2e8a"). InnerVolumeSpecName "kube-api-access-82l82". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 29 20:24:46.770160 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.770139    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62924f7c-d833-4bea-ac0d-ec24cafa2e8a" (UID: "62924f7c-d833-4bea-ac0d-ec24cafa2e8a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 29 20:24:46.868976 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.868937    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-openshift-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:46.868976 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.868971    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-maas-api-tls\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:46.868976 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:46.868983    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82l82\" (UniqueName: \"kubernetes.io/projected/62924f7c-d833-4bea-ac0d-ec24cafa2e8a-kube-api-access-82l82\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:24:47.053398 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.053305    2580 generic.go:356] "Generic (PLEG): container finished" podID="62924f7c-d833-4bea-ac0d-ec24cafa2e8a" containerID="46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4" exitCode=0
Jul 29 20:24:47.053398 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.053356    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"
Jul 29 20:24:47.053578 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.053393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" event={"ID":"62924f7c-d833-4bea-ac0d-ec24cafa2e8a","Type":"ContainerDied","Data":"46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4"}
Jul 29 20:24:47.053578 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.053451    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8" event={"ID":"62924f7c-d833-4bea-ac0d-ec24cafa2e8a","Type":"ContainerDied","Data":"59ae8fa1e1944a4c6897a6d566bc79b74e0de66d25a47775dcef8098f1949fbb"}
Jul 29 20:24:47.053578 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.053474    2580 scope.go:117] "RemoveContainer" containerID="46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4"
Jul 29 20:24:47.063461 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.063443    2580 scope.go:117] "RemoveContainer" containerID="46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4"
Jul 29 20:24:47.063727 ip-10-0-140-87 kubenswrapper[2580]: E0729 20:24:47.063712    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4\": container with ID starting with 46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4 not found: ID does not exist" containerID="46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4"
Jul 29 20:24:47.063775 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.063734    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4"} err="failed to get container status \"46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4\": rpc error: code = NotFound desc = could not find container \"46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4\": container with ID starting with 46c1f74fbeec70c9aef7d913e5eb726b2cfea89fa1c709b9d69f24a44ba80fd4 not found: ID does not exist"
Jul 29 20:24:47.078241 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.078188    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"]
Jul 29 20:24:47.080731 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.080706    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-658cad-5b7779dd88-c78n8"]
Jul 29 20:24:47.100078 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.100050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:47.171123 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.171086    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:47.496150 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:47.496108    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:24:48.717149 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:48.717111    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62924f7c-d833-4bea-ac0d-ec24cafa2e8a" path="/var/lib/kubelet/pods/62924f7c-d833-4bea-ac0d-ec24cafa2e8a/volumes"
Jul 29 20:24:52.038693 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:52.038659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:24:57.901624 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:24:57.901585    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:25:28.706218 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:28.706181    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:29.261105 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:29.261068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:35.003579 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:35.003543    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:44.274960 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:44.274917    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:44.352615 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:44.352578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:49.626257 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:49.626223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:49.697840 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:49.697807    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:49.898377 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:49.898287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:50.127216 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.127167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:25:50.243862 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.243778    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"]
Jul 29 20:25:50.244268 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244253    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed8d916b-09e7-4518-9503-0a83986eecaa" containerName="revoke-keys"
Jul 29 20:25:50.244319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244271    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed8d916b-09e7-4518-9503-0a83986eecaa" containerName="revoke-keys"
Jul 29 20:25:50.244319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244284    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy"
Jul 29 20:25:50.244319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244290    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy"
Jul 29 20:25:50.244319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244297    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62924f7c-d833-4bea-ac0d-ec24cafa2e8a" containerName="maas-api"
Jul 29 20:25:50.244319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="62924f7c-d833-4bea-ac0d-ec24cafa2e8a" containerName="maas-api"
Jul 29 20:25:50.244319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244319    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy"
Jul 29 20:25:50.244518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244325    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy"
Jul 29 20:25:50.244518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244395    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy"
Jul 29 20:25:50.244518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244403    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29997415-ff38-4059-bfd9-597b8d418a60" containerName="istio-proxy"
Jul 29 20:25:50.244518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244411    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed8d916b-09e7-4518-9503-0a83986eecaa" containerName="revoke-keys"
Jul 29 20:25:50.244518 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.244427    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="62924f7c-d833-4bea-ac0d-ec24cafa2e8a" containerName="maas-api"
Jul 29 20:25:50.248693 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.248673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:25:50.250270 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.250252    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:25:50.255237 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.255212    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"]
Jul 29 20:25:50.403935 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.403898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ec922ec-a475-4b67-87de-ae3a590386e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44\" (UID: \"6ec922ec-a475-4b67-87de-ae3a590386e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:25:50.504983 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.504877    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ec922ec-a475-4b67-87de-ae3a590386e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44\" (UID: \"6ec922ec-a475-4b67-87de-ae3a590386e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:25:50.505577 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.505551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ec922ec-a475-4b67-87de-ae3a590386e2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44\" (UID: \"6ec922ec-a475-4b67-87de-ae3a590386e2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:25:50.561522 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.561485    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:25:50.911178 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:50.911154    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"]
Jul 29 20:25:50.913074 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:25:50.913045    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ec922ec_a475_4b67_87de_ae3a590386e2.slice/crio-dc1b54703cfc4ea5de1d36bed24584753888236e371f63cc83e0efeb511737a8 WatchSource:0}: Error finding container dc1b54703cfc4ea5de1d36bed24584753888236e371f63cc83e0efeb511737a8: Status 404 returned error can't find the container with id dc1b54703cfc4ea5de1d36bed24584753888236e371f63cc83e0efeb511737a8
Jul 29 20:25:51.432437 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:51.432391    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44" event={"ID":"6ec922ec-a475-4b67-87de-ae3a590386e2","Type":"ContainerStarted","Data":"731e80dd7b0bd1659b1aa884bde7b5f37eeb3833731fdb86827acf6ee63810a3"}
Jul 29 20:25:51.432648 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:51.432458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44" event={"ID":"6ec922ec-a475-4b67-87de-ae3a590386e2","Type":"ContainerStarted","Data":"dc1b54703cfc4ea5de1d36bed24584753888236e371f63cc83e0efeb511737a8"}
Jul 29 20:25:51.446850 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:51.446623    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44" podStartSLOduration=1.446605138 podStartE2EDuration="1.446605138s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:25:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:25:51.446290548 +0000 UTC m=+3373.324144875" watchObservedRunningTime="2026-07-29 20:25:51.446605138 +0000 UTC m=+3373.324459469"
Jul 29 20:25:52.439470 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:52.439436    2580 generic.go:356] "Generic (PLEG): container finished" podID="6ec922ec-a475-4b67-87de-ae3a590386e2" containerID="731e80dd7b0bd1659b1aa884bde7b5f37eeb3833731fdb86827acf6ee63810a3" exitCode=0
Jul 29 20:25:52.439864 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:52.439503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44" event={"ID":"6ec922ec-a475-4b67-87de-ae3a590386e2","Type":"ContainerDied","Data":"731e80dd7b0bd1659b1aa884bde7b5f37eeb3833731fdb86827acf6ee63810a3"}
Jul 29 20:25:53.582709 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:53.582682    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:25:53.735459 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:53.735367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ec922ec-a475-4b67-87de-ae3a590386e2-maas-api-service-ca\") pod \"6ec922ec-a475-4b67-87de-ae3a590386e2\" (UID: \"6ec922ec-a475-4b67-87de-ae3a590386e2\") "
Jul 29 20:25:53.735761 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:53.735734    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ec922ec-a475-4b67-87de-ae3a590386e2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6ec922ec-a475-4b67-87de-ae3a590386e2" (UID: "6ec922ec-a475-4b67-87de-ae3a590386e2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:25:53.836964 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:53.836922    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ec922ec-a475-4b67-87de-ae3a590386e2-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:25:54.452558 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:54.452517    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44" event={"ID":"6ec922ec-a475-4b67-87de-ae3a590386e2","Type":"ContainerDied","Data":"dc1b54703cfc4ea5de1d36bed24584753888236e371f63cc83e0efeb511737a8"}
Jul 29 20:25:54.452746 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:54.452600    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc1b54703cfc4ea5de1d36bed24584753888236e371f63cc83e0efeb511737a8"
Jul 29 20:25:54.452746 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:25:54.452532    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"
Jul 29 20:26:00.738603 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:00.738558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:00.809018 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:00.808935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:00.914642 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:00.914610    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:00.935534 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:00.935499    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"]
Jul 29 20:26:00.938614 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:00.938588    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-259e5b-rzh44"]
Jul 29 20:26:01.014738 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:01.014649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:01.303226 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:01.303142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:02.717174 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:02.717136    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ec922ec-a475-4b67-87de-ae3a590386e2" path="/var/lib/kubelet/pods/6ec922ec-a475-4b67-87de-ae3a590386e2/volumes"
Jul 29 20:26:03.897240 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:03.897201    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:26:07.826138 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:07.826102    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:07.894349 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:07.894313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:08.088272 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.088191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:08.321260 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.321221    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:08.459760 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.459716    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"]
Jul 29 20:26:08.460308 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.460286    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ec922ec-a475-4b67-87de-ae3a590386e2" containerName="revoke-keys"
Jul 29 20:26:08.460308 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.460310    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ec922ec-a475-4b67-87de-ae3a590386e2" containerName="revoke-keys"
Jul 29 20:26:08.460505 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.460418    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ec922ec-a475-4b67-87de-ae3a590386e2" containerName="revoke-keys"
Jul 29 20:26:08.465098 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.465075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:08.467613 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.467590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-vt6mz\""
Jul 29 20:26:08.469797 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.469776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"]
Jul 29 20:26:08.486601 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.486567    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11936d-b25b-4b4f-a7d1-33983767a7cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx\" (UID: \"0a11936d-b25b-4b4f-a7d1-33983767a7cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:08.587259 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.587218    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11936d-b25b-4b4f-a7d1-33983767a7cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx\" (UID: \"0a11936d-b25b-4b4f-a7d1-33983767a7cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:08.587837 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.587817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11936d-b25b-4b4f-a7d1-33983767a7cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx\" (UID: \"0a11936d-b25b-4b4f-a7d1-33983767a7cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:08.778780 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.778664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:08.921672 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:08.921645    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"]
Jul 29 20:26:08.923421 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:26:08.923394    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a11936d_b25b_4b4f_a7d1_33983767a7cd.slice/crio-530b3da7f412204112bfa031028d9a1b691d47af3e098cdf16ebeae134114e61 WatchSource:0}: Error finding container 530b3da7f412204112bfa031028d9a1b691d47af3e098cdf16ebeae134114e61: Status 404 returned error can't find the container with id 530b3da7f412204112bfa031028d9a1b691d47af3e098cdf16ebeae134114e61
Jul 29 20:26:09.538603 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:09.538509    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx" event={"ID":"0a11936d-b25b-4b4f-a7d1-33983767a7cd","Type":"ContainerStarted","Data":"99e8538a257759562046c2f6adc24495930c7410fd8b3cfec12eb45c45e00fda"}
Jul 29 20:26:09.538603 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:09.538562    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx" event={"ID":"0a11936d-b25b-4b4f-a7d1-33983767a7cd","Type":"ContainerStarted","Data":"530b3da7f412204112bfa031028d9a1b691d47af3e098cdf16ebeae134114e61"}
Jul 29 20:26:09.557773 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:09.557471    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx" podStartSLOduration=1.557454408 podStartE2EDuration="1.557454408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:26:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:26:09.557061418 +0000 UTC m=+3391.434915747" watchObservedRunningTime="2026-07-29 20:26:09.557454408 +0000 UTC m=+3391.435308736"
Jul 29 20:26:10.545902 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:10.545868    2580 generic.go:356] "Generic (PLEG): container finished" podID="0a11936d-b25b-4b4f-a7d1-33983767a7cd" containerID="99e8538a257759562046c2f6adc24495930c7410fd8b3cfec12eb45c45e00fda" exitCode=0
Jul 29 20:26:10.546315 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:10.545926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx" event={"ID":"0a11936d-b25b-4b4f-a7d1-33983767a7cd","Type":"ContainerDied","Data":"99e8538a257759562046c2f6adc24495930c7410fd8b3cfec12eb45c45e00fda"}
Jul 29 20:26:11.696791 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:11.696763    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:11.719662 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:11.719632    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11936d-b25b-4b4f-a7d1-33983767a7cd-maas-api-service-ca\") pod \"0a11936d-b25b-4b4f-a7d1-33983767a7cd\" (UID: \"0a11936d-b25b-4b4f-a7d1-33983767a7cd\") "
Jul 29 20:26:11.720090 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:11.720063    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0a11936d-b25b-4b4f-a7d1-33983767a7cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0a11936d-b25b-4b4f-a7d1-33983767a7cd" (UID: "0a11936d-b25b-4b4f-a7d1-33983767a7cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 29 20:26:11.821319 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:11.821245    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0a11936d-b25b-4b4f-a7d1-33983767a7cd-maas-api-service-ca\") on node \"ip-10-0-140-87.ec2.internal\" DevicePath \"\""
Jul 29 20:26:12.558781 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:12.558744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx" event={"ID":"0a11936d-b25b-4b4f-a7d1-33983767a7cd","Type":"ContainerDied","Data":"530b3da7f412204112bfa031028d9a1b691d47af3e098cdf16ebeae134114e61"}
Jul 29 20:26:12.558781 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:12.558778    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"
Jul 29 20:26:12.559018 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:12.558794    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="530b3da7f412204112bfa031028d9a1b691d47af3e098cdf16ebeae134114e61"
Jul 29 20:26:18.933285 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:18.933233    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:18.933943 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:18.933901    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"]
Jul 29 20:26:18.936142 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:18.936121    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-7d4a80-29755935-jcgpx"]
Jul 29 20:26:19.009703 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:19.009642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:19.071713 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:19.071671    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:19.127490 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:19.127452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"]
Jul 29 20:26:19.129297 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:19.129271    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:19.129871 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:19.129847    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-7d4a80-b4cdx"]
Jul 29 20:26:19.509083 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:19.509053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:20.056651 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:20.056616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:20.716203 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:20.716169    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0a11936d-b25b-4b4f-a7d1-33983767a7cd" path="/var/lib/kubelet/pods/0a11936d-b25b-4b4f-a7d1-33983767a7cd/volumes"
Jul 29 20:26:20.716496 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:20.716482    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c8fa379-840b-4445-b8d1-68983ea62be8" path="/var/lib/kubelet/pods/6c8fa379-840b-4445-b8d1-68983ea62be8/volumes"
Jul 29 20:26:22.096364 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:22.096328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:26:25.921962 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:25.921919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:26.022036 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:26.021978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:26.185443 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:26.185358    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:26.405043 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:26.404977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:36.994270 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:36.994239    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"]
Jul 29 20:26:36.994758 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:36.994503    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:36.996421 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:36.996401    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-1a1eaa-29755935-vz2gp"]
Jul 29 20:26:37.093303 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:37.093272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:37.193133 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:37.193095    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:37.293553 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:37.293463    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:37.586664 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:37.586583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:38.716624 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:38.716591    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="707504db-4880-42c3-8198-7a7d26dda73e" path="/var/lib/kubelet/pods/707504db-4880-42c3-8198-7a7d26dda73e/volumes"
Jul 29 20:26:39.169267 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:39.169234    2580 scope.go:117] "RemoveContainer" containerID="d61f4bb8dd323fe7433e5c058f3aade3b262f531c56f3d3597b0b1ea373caa82"
Jul 29 20:26:39.178888 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:39.178864    2580 scope.go:117] "RemoveContainer" containerID="745694706eb463cf2b598ab1f4444e4d027a7b68fa10db1a98690ad79684e2a2"
Jul 29 20:26:39.694740 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:39.694706    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-dvfvg"]
Jul 29 20:26:44.009778 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:44.009739    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:44.070216 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:44.070182    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:44.265958 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:44.265876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:44.489275 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:44.489236    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:26:44.898443 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:44.898399    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bbcb9d87-8pf22_2665e292-089d-441e-b40d-aa42429ec3fc/authorino/0.log"
Jul 29 20:26:50.048177 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:50.048151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vslbl_796bc525-917d-4964-9056-4849fa4a9fbf/manager/0.log"
Jul 29 20:26:50.163141 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:50.163109    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w9pl6_a85212a3-270a-4dea-9207-98b2ffd69cd0/manager/0.log"
Jul 29 20:26:50.531576 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:50.531546    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8t7ch_f00a6dce-0179-4df2-9b80-8d22df1b29b7/manager/0.log"
Jul 29 20:26:50.765267 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:50.765240    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d75b7b96-q7mlp_e5d77ff4-9f8b-4a93-91c9-f1b30213a39c/copy-manifests/0.log"
Jul 29 20:26:50.771203 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:50.771179    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d75b7b96-q7mlp_e5d77ff4-9f8b-4a93-91c9-f1b30213a39c/manager/0.log"
Jul 29 20:26:51.582682 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:51.582654    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29755935-7zl62_1a440564-314c-45f9-9fb3-28c6f7e1dc61/cleanup/0.log"
Jul 29 20:26:51.693376 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:51.693344    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-lj9jb_272e1ad1-770c-4d30-9942-3b378ea28713/postgres/0.log"
Jul 29 20:26:52.936916 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:52.936874    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bbcb9d87-8pf22_2665e292-089d-441e-b40d-aa42429ec3fc/authorino/0.log"
Jul 29 20:26:53.055979 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:53.055948    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8mzbx_3cec8f23-6e0e-4886-bc86-1c549fabd341/manager/0.log"
Jul 29 20:26:53.182682 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:53.182651    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w5b9f_4eb9318a-4701-4713-9eb4-273d5c9c5318/manager/0.log"
Jul 29 20:26:53.403078 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:53.403047    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-54w2z_7aeac165-0eb7-4d3a-b6c6-a5e820016ae3/registry-server/0.log"
Jul 29 20:26:53.632511 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:53.632483    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-dvfvg_56c9f00e-289b-4e20-a364-c4a189c6c469/limitador/0.log"
Jul 29 20:26:53.749227 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:53.749150    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8hlt2_53c58309-bd86-4cfb-9b0d-db48f15fbae5/manager/0.log"
Jul 29 20:26:54.408315 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:54.408285    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cc65fd5df-xng9r_fb3bfa3a-a3e6-4290-b230-5a906f026bcb/kube-auth-proxy/0.log"
Jul 29 20:26:54.519408 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:54.519375    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/1.log"
Jul 29 20:26:54.965632 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:54.965600    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66df6dd487-2zjcx_c83babd6-f73c-4bdf-8ee2-bf8ade50ae73/router/0.log"
Jul 29 20:26:55.629190 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:55.629154    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fdhlsx_6c1cbe5d-ba5a-42da-82a2-f4355876d802/main/0.log"
Jul 29 20:26:55.735156 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:26:55.735111    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-gnp4s_a36710e8-c8b8-4a86-93c1-1a1defa6fda0/main/0.log"
Jul 29 20:27:00.974223 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.974190    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cq5dz/must-gather-v68gk"]
Jul 29 20:27:00.974634 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.974621    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a11936d-b25b-4b4f-a7d1-33983767a7cd" containerName="revoke-keys"
Jul 29 20:27:00.974682 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.974669    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a11936d-b25b-4b4f-a7d1-33983767a7cd" containerName="revoke-keys"
Jul 29 20:27:00.974784 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.974754    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a11936d-b25b-4b4f-a7d1-33983767a7cd" containerName="revoke-keys"
Jul 29 20:27:00.978506 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.978474    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:00.980632 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.980609    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-cq5dz\"/\"default-dockercfg-8c6m4\""
Jul 29 20:27:00.980632 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.980631    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cq5dz\"/\"kube-root-ca.crt\""
Jul 29 20:27:00.980797 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.980633    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-cq5dz\"/\"openshift-service-ca.crt\""
Jul 29 20:27:00.987186 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.987162    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq5dz/must-gather-v68gk"]
Jul 29 20:27:00.989523 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:00.989500    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:27:01.090247 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.090213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:27:01.099345 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.099317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmgmj\" (UniqueName: \"kubernetes.io/projected/ddb83038-fb66-4050-bfb5-667a6f2b4962-kube-api-access-nmgmj\") pod \"must-gather-v68gk\" (UID: \"ddb83038-fb66-4050-bfb5-667a6f2b4962\") " pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.099464 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.099362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ddb83038-fb66-4050-bfb5-667a6f2b4962-must-gather-output\") pod \"must-gather-v68gk\" (UID: \"ddb83038-fb66-4050-bfb5-667a6f2b4962\") " pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.200022 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.199962    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmgmj\" (UniqueName: \"kubernetes.io/projected/ddb83038-fb66-4050-bfb5-667a6f2b4962-kube-api-access-nmgmj\") pod \"must-gather-v68gk\" (UID: \"ddb83038-fb66-4050-bfb5-667a6f2b4962\") " pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.200235 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.200046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ddb83038-fb66-4050-bfb5-667a6f2b4962-must-gather-output\") pod \"must-gather-v68gk\" (UID: \"ddb83038-fb66-4050-bfb5-667a6f2b4962\") " pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.200348 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.200334    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/ddb83038-fb66-4050-bfb5-667a6f2b4962-must-gather-output\") pod \"must-gather-v68gk\" (UID: \"ddb83038-fb66-4050-bfb5-667a6f2b4962\") " pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.212154 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.212127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmgmj\" (UniqueName: \"kubernetes.io/projected/ddb83038-fb66-4050-bfb5-667a6f2b4962-kube-api-access-nmgmj\") pod \"must-gather-v68gk\" (UID: \"ddb83038-fb66-4050-bfb5-667a6f2b4962\") " pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.288744 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.288661    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq5dz/must-gather-v68gk"
Jul 29 20:27:01.430427 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.430396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq5dz/must-gather-v68gk"]
Jul 29 20:27:01.432045 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:27:01.431986    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddb83038_fb66_4050_bfb5_667a6f2b4962.slice/crio-66f22193718ccecb307d502e7bcd6253c4dcf57ebad00b0194af4ab6663b1348 WatchSource:0}: Error finding container 66f22193718ccecb307d502e7bcd6253c4dcf57ebad00b0194af4ab6663b1348: Status 404 returned error can't find the container with id 66f22193718ccecb307d502e7bcd6253c4dcf57ebad00b0194af4ab6663b1348
Jul 29 20:27:01.433865 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.433847    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 29 20:27:01.847337 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:01.847302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq5dz/must-gather-v68gk" event={"ID":"ddb83038-fb66-4050-bfb5-667a6f2b4962","Type":"ContainerStarted","Data":"66f22193718ccecb307d502e7bcd6253c4dcf57ebad00b0194af4ab6663b1348"}
Jul 29 20:27:02.858454 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:02.858403    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq5dz/must-gather-v68gk" event={"ID":"ddb83038-fb66-4050-bfb5-667a6f2b4962","Type":"ContainerStarted","Data":"568515637c51dee3c566f49c62be31e4f3dac59f90c1dad7f7a1e9bd5f5c7d73"}
Jul 29 20:27:02.858454 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:02.858456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq5dz/must-gather-v68gk" event={"ID":"ddb83038-fb66-4050-bfb5-667a6f2b4962","Type":"ContainerStarted","Data":"2becea471e3e863729c2d00ca09e3f76b6893c82c8a26bfc08fc5a9a2ef299ac"}
Jul 29 20:27:02.875369 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:02.875301    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cq5dz/must-gather-v68gk" podStartSLOduration=2.127950405 podStartE2EDuration="2.875278419s" totalImagesPullingTime="747.328014ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-29 20:27:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:27:02.874579141 +0000 UTC m=+3444.752433474" watchObservedRunningTime="2026-07-29 20:27:02.875278419 +0000 UTC m=+3444.753132748"
Jul 29 20:27:03.785630 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:03.785598    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ffjxb_6b8bf68c-0fd8-442c-af0b-90e8fbe11f84/global-pull-secret-syncer/0.log"
Jul 29 20:27:03.912287 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:03.912252    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log"
Jul 29 20:27:04.072131 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:04.072054    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-87.ec2.internal_ab41b11aff5e6bd368de8ec2efaf8430/haproxy/0.log"
Jul 29 20:27:09.735925 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:09.735699    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-bbcb9d87-8pf22_2665e292-089d-441e-b40d-aa42429ec3fc/authorino/0.log"
Jul 29 20:27:09.769015 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:09.768809    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-8mzbx_3cec8f23-6e0e-4886-bc86-1c549fabd341/manager/0.log"
Jul 29 20:27:09.808573 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:09.808538    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-w5b9f_4eb9318a-4701-4713-9eb4-273d5c9c5318/manager/0.log"
Jul 29 20:27:09.891741 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:09.891702    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-54w2z_7aeac165-0eb7-4d3a-b6c6-a5e820016ae3/registry-server/0.log"
Jul 29 20:27:10.163624 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:10.163593    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-dvfvg_56c9f00e-289b-4e20-a364-c4a189c6c469/limitador/0.log"
Jul 29 20:27:10.251276 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:10.251204    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-8hlt2_53c58309-bd86-4cfb-9b0d-db48f15fbae5/manager/0.log"
Jul 29 20:27:11.835912 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:11.835879    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/alertmanager/0.log"
Jul 29 20:27:11.860775 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:11.860746    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/config-reloader/0.log"
Jul 29 20:27:11.887269 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:11.887215    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/kube-rbac-proxy-web/0.log"
Jul 29 20:27:11.946055 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:11.945968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/kube-rbac-proxy/0.log"
Jul 29 20:27:11.995163 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:11.995138    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/kube-rbac-proxy-metric/0.log"
Jul 29 20:27:12.038129 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.038101    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/prom-label-proxy/0.log"
Jul 29 20:27:12.063824 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.063788    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_507ab561-0f67-4cb0-9c6a-6d06cc9c22c2/init-config-reloader/0.log"
Jul 29 20:27:12.112865 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.112775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-rftbz_efb86fd8-4782-4e7d-b732-52987e85c001/cluster-monitoring-operator/0.log"
Jul 29 20:27:12.143903 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.143871    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-c8dx7_f9deef4a-c592-4e2a-bd38-cc3ed62c5f67/kube-state-metrics/0.log"
Jul 29 20:27:12.164278 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.164250    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-c8dx7_f9deef4a-c592-4e2a-bd38-cc3ed62c5f67/kube-rbac-proxy-main/0.log"
Jul 29 20:27:12.192470 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.192400    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56d9f944b4-c8dx7_f9deef4a-c592-4e2a-bd38-cc3ed62c5f67/kube-rbac-proxy-self/0.log"
Jul 29 20:27:12.250945 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.250905    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-l9fkk_24705fb5-d1aa-4b14-a8f3-fd6a83439acd/monitoring-plugin/0.log"
Jul 29 20:27:12.442139 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.442106    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcp8p_6f902dac-3eaa-48f2-a226-7f392a185571/node-exporter/0.log"
Jul 29 20:27:12.464418 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.464386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcp8p_6f902dac-3eaa-48f2-a226-7f392a185571/kube-rbac-proxy/0.log"
Jul 29 20:27:12.488091 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.488055    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hcp8p_6f902dac-3eaa-48f2-a226-7f392a185571/init-textfile/0.log"
Jul 29 20:27:12.847927 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.847853    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-qd92r_041a18b2-f783-458b-adbf-4e8abef2fd61/prometheus-operator-admission-webhook/0.log"
Jul 29 20:27:12.882036 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.881984    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f9dbf67db-bkp9l_63c2f63f-85d3-40a9-ae85-31190b462604/telemeter-client/0.log"
Jul 29 20:27:12.905174 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.905147    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f9dbf67db-bkp9l_63c2f63f-85d3-40a9-ae85-31190b462604/reload/0.log"
Jul 29 20:27:12.928672 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:12.928641    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5f9dbf67db-bkp9l_63c2f63f-85d3-40a9-ae85-31190b462604/kube-rbac-proxy/0.log"
Jul 29 20:27:14.211930 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.211892    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"]
Jul 29 20:27:14.219335 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.219303    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.226257 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.226131    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"]
Jul 29 20:27:14.355384 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.355343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-podres\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.355604 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.355538    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-lib-modules\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.355604 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.355585    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-sys\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.355727 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.355619    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fhdd\" (UniqueName: \"kubernetes.io/projected/5739f749-61c7-48c5-92c0-b621684cae52-kube-api-access-7fhdd\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.355727 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.355644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-proc\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.456634 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.456589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-sys\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.456838 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.456642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fhdd\" (UniqueName: \"kubernetes.io/projected/5739f749-61c7-48c5-92c0-b621684cae52-kube-api-access-7fhdd\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.456838 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.456670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-proc\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.456838 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.456773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-podres\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.457018 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.456854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-lib-modules\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.457079 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.457060    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-lib-modules\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.457139 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.457122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-sys\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.457471 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.457448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-proc\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.457574 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.457557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/5739f749-61c7-48c5-92c0-b621684cae52-podres\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.466031 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.465935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fhdd\" (UniqueName: \"kubernetes.io/projected/5739f749-61c7-48c5-92c0-b621684cae52-kube-api-access-7fhdd\") pod \"perf-node-gather-daemonset-hllm5\" (UID: \"5739f749-61c7-48c5-92c0-b621684cae52\") " pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.534700 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.534668    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.731271 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.731068    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"]
Jul 29 20:27:14.735588 ip-10-0-140-87 kubenswrapper[2580]: W0729 20:27:14.735517    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod5739f749_61c7_48c5_92c0_b621684cae52.slice/crio-50991da865cd9f3c61709f72ecbeefb99560794a1da00f4f26fee5311e210b98 WatchSource:0}: Error finding container 50991da865cd9f3c61709f72ecbeefb99560794a1da00f4f26fee5311e210b98: Status 404 returned error can't find the container with id 50991da865cd9f3c61709f72ecbeefb99560794a1da00f4f26fee5311e210b98
Jul 29 20:27:14.962478 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.962438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5" event={"ID":"5739f749-61c7-48c5-92c0-b621684cae52","Type":"ContainerStarted","Data":"77d03e1aff32101c7686976a20574f54073cfa3f19bdc8a8d3ac6929308b6011"}
Jul 29 20:27:14.962755 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.962504    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5" event={"ID":"5739f749-61c7-48c5-92c0-b621684cae52","Type":"ContainerStarted","Data":"50991da865cd9f3c61709f72ecbeefb99560794a1da00f4f26fee5311e210b98"}
Jul 29 20:27:14.962755 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.962538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:14.978643 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.978579    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5" podStartSLOduration=0.978559704 podStartE2EDuration="978.559704ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-29 20:27:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-29 20:27:14.976215311 +0000 UTC m=+3456.854069640" watchObservedRunningTime="2026-07-29 20:27:14.978559704 +0000 UTC m=+3456.856414035"
Jul 29 20:27:14.979450 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.979419    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/1.log"
Jul 29 20:27:14.986623 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:14.986519    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-jrc7p_8b6c4c13-22b0-41c5-bcb9-ce0a18589113/console-operator/2.log"
Jul 29 20:27:15.478520 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:15.478491    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c6967c554-ntwd5_1672f85e-896b-4c77-b824-710a1934998c/console/0.log"
Jul 29 20:27:16.127891 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:16.127858    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-clg2w_342114ba-5b4a-43af-9260-78a7e1013d93/volume-data-source-validator/0.log"
Jul 29 20:27:17.320899 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:17.320853    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pclnw_f9f66716-3a4b-4276-bef5-7575195acbfc/dns/0.log"
Jul 29 20:27:17.344253 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:17.344227    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pclnw_f9f66716-3a4b-4276-bef5-7575195acbfc/kube-rbac-proxy/0.log"
Jul 29 20:27:17.398371 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:17.398342    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j5vzw_464c7d4e-5049-415c-bf60-3bab6ae5429c/dns-node-resolver/0.log"
Jul 29 20:27:18.081468 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:18.081423    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-pg9vq_4ffbdef9-85a9-4272-b1ba-dbe67cb24dd4/node-ca/0.log"
Jul 29 20:27:19.293033 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:19.292987    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cc65fd5df-xng9r_fb3bfa3a-a3e6-4290-b230-5a906f026bcb/kube-auth-proxy/0.log"
Jul 29 20:27:19.321781 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:19.321746    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/0.log"
Jul 29 20:27:19.357700 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:19.357671    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/1.log"
Jul 29 20:27:19.458402 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:19.458371    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66df6dd487-2zjcx_c83babd6-f73c-4bdf-8ee2-bf8ade50ae73/router/0.log"
Jul 29 20:27:20.331516 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:20.331483    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-n88rq_e3b0bec5-445e-4fa4-bde1-86becceb24a8/serve-healthcheck-canary/0.log"
Jul 29 20:27:20.853200 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:20.853168    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-dlrtg_588349d1-be21-4927-b936-0ba5f6e725b3/insights-operator/0.log"
Jul 29 20:27:20.853766 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:20.853744    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-dlrtg_588349d1-be21-4927-b936-0ba5f6e725b3/insights-operator/1.log"
Jul 29 20:27:20.983450 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:20.983419    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-cq5dz/perf-node-gather-daemonset-hllm5"
Jul 29 20:27:21.029776 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:21.029745    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vrvjf_0db0227f-3080-4a56-8950-bca5e9aeeea8/kube-rbac-proxy/0.log"
Jul 29 20:27:21.053261 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:21.053230    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vrvjf_0db0227f-3080-4a56-8950-bca5e9aeeea8/exporter/0.log"
Jul 29 20:27:21.078019 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:21.077964    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vrvjf_0db0227f-3080-4a56-8950-bca5e9aeeea8/extractor/0.log"
Jul 29 20:27:23.385694 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:23.385627    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-vslbl_796bc525-917d-4964-9056-4849fa4a9fbf/manager/0.log"
Jul 29 20:27:23.678017 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:23.677596    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-w9pl6_a85212a3-270a-4dea-9207-98b2ffd69cd0/manager/0.log"
Jul 29 20:27:24.077848 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:24.077366    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-8t7ch_f00a6dce-0179-4df2-9b80-8d22df1b29b7/manager/0.log"
Jul 29 20:27:24.164317 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:24.164290    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d75b7b96-q7mlp_e5d77ff4-9f8b-4a93-91c9-f1b30213a39c/manager/0.log"
Jul 29 20:27:24.189631 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:24.189592    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5d75b7b96-q7mlp_e5d77ff4-9f8b-4a93-91c9-f1b30213a39c/copy-manifests/0.log"
Jul 29 20:27:25.062404 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:25.062379    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cc65fd5df-xng9r_fb3bfa3a-a3e6-4290-b230-5a906f026bcb/kube-auth-proxy/0.log"
Jul 29 20:27:25.081427 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:25.081395    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/0.log"
Jul 29 20:27:25.117301 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:25.117266    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/1.log"
Jul 29 20:27:25.186782 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:25.186750    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66df6dd487-2zjcx_c83babd6-f73c-4bdf-8ee2-bf8ade50ae73/router/0.log"
Jul 29 20:27:25.674122 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:25.674089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-kcpzv_b9d37d8b-f5f9-4265-828d-dec5586cf62a/openshift-lws-operator/0.log"
Jul 29 20:27:26.824780 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:26.824744    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6cc65fd5df-xng9r_fb3bfa3a-a3e6-4290-b230-5a906f026bcb/kube-auth-proxy/0.log"
Jul 29 20:27:26.845250 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:26.845217    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/0.log"
Jul 29 20:27:26.879913 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:26.879881    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-7rzcx_4f016396-42e6-4348-a688-dc65be237fa3/istio-proxy/1.log"
Jul 29 20:27:26.950481 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:26.950455    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66df6dd487-2zjcx_c83babd6-f73c-4bdf-8ee2-bf8ade50ae73/router/0.log"
Jul 29 20:27:31.191980 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:31.191949    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns_7894863b-2223-4519-b749-851b66facc7b/kube-storage-version-migrator-operator/1.log"
Jul 29 20:27:31.193029 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:31.193009    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-zv9ns_7894863b-2223-4519-b749-851b66facc7b/kube-storage-version-migrator-operator/0.log"
Jul 29 20:27:31.301210 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:31.301168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024
Jul 29 20:27:31.396467 ip-10-0-140-87 kubenswrapper[2580]: I0729 20:27:31.396430    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-7fqz7_a6aad7a7-132c-49e6-91ef-a32087911aee/konnectivity-agent/0.log" limit=1024